-- Logs begin at Fri 2022-07-01 02:34:39 UTC, end at Fri 2022-07-01 04:27:28 UTC. -- Jul 01 03:41:50.854744 dv-cinder-846842-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 01 03:41:50.862603 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 1 03:41:50 2022] *** Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: nodename: dv-cinder-846842-8-iscsi Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: machine: x86_64 Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: clock source: unix Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: pcre jit disabled Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: detected number of CPU cores: 4 Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: current working directory: / Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: detected binary path: /usr/bin/uwsgi-core Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: your processes number limit is 47680 Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: your memory page size is 4096 bytes Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: detected max file descriptor number: 2048 Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: lock engine: pthread robust mutexes Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: thunder lock: enabled Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 01 03:41:50.868870 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 01 03:41:50.913198 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: Python main interpreter initialized at 0x55f00652d9c0 Jul 01 03:41:50.913198 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: python threads support enabled Jul 01 03:41:50.913198 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: your server socket listen backlog is limited to 100 connections Jul 01 03:41:50.913198 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: your mercy for graceful operations on workers is 90 seconds Jul 01 03:41:50.916257 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: mapped 403077 bytes (393 KB) for 2 cores Jul 01 03:41:50.916257 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: *** Operational MODE: preforking *** Jul 01 03:41:50.916257 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: *** uWSGI is running in multiple interpreter mode *** Jul 01 03:41:50.916257 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: spawned uWSGI master process (pid: 80563) Jul 01 03:41:50.916257 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: spawned uWSGI worker 1 (pid: 80576, cores: 1) Jul 01 03:41:50.916257 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: spawned uWSGI worker 2 (pid: 80577, cores: 1) Jul 01 03:41:50.916257 dv-cinder-846842-8-iscsi devstack@keystone.service[80563]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 01 03:41:50.914151 dv-cinder-846842-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 01 03:41:56.446458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 03:41:56.446458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: from cryptography.utils import int_from_bytes Jul 01 03:41:56.447341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 03:41:56.447341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: from cryptography.utils import int_from_bytes Jul 01 03:41:56.447767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 03:41:56.447767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: from cryptography.utils import int_from_bytes Jul 01 03:41:56.448224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 03:41:56.448224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: from cryptography.utils import int_from_bytes Jul 01 03:41:56.501786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 01 03:41:56.501988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 01 03:41:56.502081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 01 03:41:56.502178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 01 03:41:56.502260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] command line args: [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 01 03:41:56.502344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] command line args: [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 01 03:41:56.502423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 01 03:41:56.502501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 01 03:41:56.502578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ================================================================================ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 01 03:41:56.502655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ================================================================================ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 01 03:41:56.502809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] admin_token = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.502898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] admin_token = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] config_dir = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] config_dir = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] config_source = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503568 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] config_source = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] debug = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] debug = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.503781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.504154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.504299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.504406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.504509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.504611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.504720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.504833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.504939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_config_append = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_config_append = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_dir = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_dir = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.505947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_file = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_file = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506473 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.506690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.507629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.507748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.507841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.507922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508125 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.508923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.509935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] publish_errors = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] publish_errors = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.510950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] standard_threads = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] standard_threads = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_journal = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.511939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_json = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_journal = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_stderr = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_json = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_syslog = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_stderr = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_syslog = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 03:41:56.512814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.512983 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.513112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.513229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.513312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.513469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.513605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.513722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.513859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.513943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.514015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.514190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.514275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.514431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.514533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.514674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.514767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.514917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.515001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.515207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.515318 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.515487 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.515571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.515726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.515810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.515978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.516058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.516220 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.516304 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.516464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.516541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.516702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.516782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.516941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.517018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.517196 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.517273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.517444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.517525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.517683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.517765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.517921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.518006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.518177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.518295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.518551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.518630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.518801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.518887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.519035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.519113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.519290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.519371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.519528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.519621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.519763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.519846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.520005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.520089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.520254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.520339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.520488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.520575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.520887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.521902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.522925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.523070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.523158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.523321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.523413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.523556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.523633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.523802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.523881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.524035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.524123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.524274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.524397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.524506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.524594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.524725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.524844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.525001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.525125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.525236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.525338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.525501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.525597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.525744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.525858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.526021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.526129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.526293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.526969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.527103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.527216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.527423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.527556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.527708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.527873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.528008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.528116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.528224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.528357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.528475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.528594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.528705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.528828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.529021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.529186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.529328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.529475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.529606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.529745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.529873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.529999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.530241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.530370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.530493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.530617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.530742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.530868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.531006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.531147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.532088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.532221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.532321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.532479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.532672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.532784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.532870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.532945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533250 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.533949 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534890 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.534961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535031 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535352 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.535952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.536998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.537175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.537256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.537436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.537538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.537704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.537782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.537938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.external = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.external = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.password = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.538983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.539146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.token = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.539294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.539410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.539491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.password = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.539648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.539736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.token = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.539892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.539971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.540121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.540200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.540370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.540452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.540602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.540680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.540839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.540918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.541078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.541159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.541319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.541397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.541561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.541645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.541807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.541890 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.542093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.542692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.542859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.542944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.543097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.543177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.543381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.543472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.543661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.543745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.543942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.544025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.544206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.544287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.544449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.544538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.544705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.544788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.544980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.545058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.545225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.545304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.545484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.545566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.545722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.545802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.545962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.546041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.546207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.546451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.546530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.546681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.546767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.546927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.547008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.547158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.547269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.547441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.547569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.547679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.547818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.547934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.548993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.549142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.549237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.549468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.549562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.549659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.549814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.549896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.550059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.550138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.550300 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.550381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.550552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.550634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.550793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.550897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551031 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.551999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.552080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.552234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.552315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.552484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.552563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.552730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.552810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.552978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.553057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.553225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.553309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.553464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.553552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.553696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.553775 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.553933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.554951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.555109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.555190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.555372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.555451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.555616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.555959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.556125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.556204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.556357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.556444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.556598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.556677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.556841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.556920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.557070 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.557158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.557309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.557389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.557537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.557617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.557802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.557891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.558046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.558126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.558280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.558361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.558516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.558603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.558756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.558835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.559980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.560071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.560214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.560294 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.560448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.560576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.560731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.560820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.560982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.561062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.561211 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.561290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.561451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.561539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.561690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.561770 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.561923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.562969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.563110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.563189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.563367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.563486 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.563607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.566448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.566536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.566610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.566683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.566755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.566827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.566913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.566989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.567955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.568980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.569936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.570950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.571911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.572062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.572142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] role.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.572312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.572397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] role.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.572545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.572633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.572785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.572864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.573026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] role.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.573105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.573286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.573377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] role.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.573529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.573633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.573775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.573856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.574021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.574114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.574267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.574348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.574513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.574593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.574762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.574859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.575009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.575090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.575278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.575362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.575655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.575756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.575904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.575985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.576157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.576347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.576427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.576583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.576665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.576832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.576912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.577081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.577297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.577385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.577536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.577638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.577782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.577863 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.578980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.579060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.579210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.579304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.579482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.579581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.579726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.579799 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.579959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.580972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.581125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.581204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.581370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.581451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.581603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.581690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.581852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.581932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.582114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.582201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.582350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.582433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.582595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.582676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.582839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.582920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.583084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.583167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.583344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.583428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.583583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.583663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.583819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.583900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.584063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.584144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.584308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.584387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.584536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.584616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.584776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.584855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.585024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.585103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.585276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.585357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.585508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.585599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.585754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.585835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.586992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.587071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.587246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.587330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.587492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.587571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.587727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.587823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.587965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.588035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.588199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.588277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.588443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.588532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.588675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.588754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.589435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.589525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.589670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.589759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.589911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.589991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.590146 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.590225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.590296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 01 03:41:56.590381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.590613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.590857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.591095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 03:41:56.591288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 01 03:41:56.621906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.622114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.622336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.622422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.622594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.622818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.631197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.631493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.661437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.661665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.661762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.661975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.816296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.817004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.866932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.867170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.872416 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.872753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.943345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.943578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.943763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.943966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.944141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.944221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.944295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.944369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.944453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.944528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.944602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.944797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.983475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.983649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.983740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.983820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.983896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.984894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.995917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:56.996477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.008581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.008822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.009899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.010029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.010190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.010407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 03:41:57.028223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.028439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 03:41:57.028769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.028883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.028967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 03:41:57.029043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 03:41:57.029222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.029306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.029477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.029562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 03:41:57.029731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.029813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.029985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.030067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.030241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.030501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.040813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.041088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.041353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.041477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.041638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.041765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.041971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.042406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.049954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.050541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.053255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.053535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.053815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.053906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.054093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.054177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.054372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.054454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.054630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.054720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.054894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.054975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.055166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.055268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.055459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.055552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.055981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.056068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.056244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.056325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.056516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.056785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.083019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.083993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.084655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.084931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.085669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.085953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.090789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.091063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.091485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.091789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.092070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.092468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.098735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.099012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.099761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.100046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.103517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.103802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.104085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.104370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.104547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.104688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.104817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.104944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.105091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.105209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.105417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.105531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.105692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.105965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.106232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.106501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.124099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.124377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.125142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.125422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.129048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.130034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.132691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.132987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.133686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.133983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.138604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.139655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.142199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.143271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.145873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.146166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.146452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.146735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.146932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.147016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.147252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.147339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.147544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.147834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.148129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.148426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.166738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.167055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.167349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.167615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.167995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.168285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.168553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.168821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.175482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.176771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.177194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.177468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.178508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.178786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.181063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.181520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.181816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.182096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.182379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.182490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.182799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.183078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.183382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.183659 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.195717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.195993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.196282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.196563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.196844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.196936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.197153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.197433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.197716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.198008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.199036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.199363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.200245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.200551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.217827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.218919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.221096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.222214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.224422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.225566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.226144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.227332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.227808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.228088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.228368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.228960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.229249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.229554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.235795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.237032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.237347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.238638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.240588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.240885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.241283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.241562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.241876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.242167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.242566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.242848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.251167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.251540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.251822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.252577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.252867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.253149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.259688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.259966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.260244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.260517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.261446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.261812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.261904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 03:41:57.262096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.262178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.262371 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.262465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.262714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.262811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.262944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.263030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.263261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.263358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.263521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.263603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.263788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.263868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.264056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.264139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.264411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.264693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.264968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.265244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.265513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 03:41:57.301699 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f00652d9c0 pid: 80576 (default app) Jul 01 03:41:57.303462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f00652d9c0 pid: 80577 (default app) Jul 01 03:41:57.305165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c8c8f-4738-4695-b145-205493496f9b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:41:57.305453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c8c8f-4738-4695-b145-205493496f9b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:41:57.305982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c8c8f-4738-4695-b145-205493496f9b None None] PATH_INFO: `/v3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:41:57.310740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Fri Jul 1 03:41:57 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:05.954307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017ecd4-fd2a-4ece-a75d-540d82b7bc71 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:05.954844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017ecd4-fd2a-4ece-a75d-540d82b7bc71 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:05.954926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017ecd4-fd2a-4ece-a75d-540d82b7bc71 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:05.976464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:42:05.976610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:05.976610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:05.976610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:06.034301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: INFO dbcounter [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] Registered counter for database keystone Jul 01 03:42:06.056150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG oslo_db.sqlalchemy.engines [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80576) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 01 03:42:06.063204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writer thread running {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 01 03:42:06.244075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG passlib.handlers.bcrypt [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80576) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 01 03:42:06.244408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG passlib.handlers.bcrypt [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80576) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 01 03:42:06.299837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:06.301495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:06.423013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 01 03:42:06.628258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017ecd4-fd2a-4ece-a75d-540d82b7bc71 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:06.699315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: INFO dbcounter [None req-5017ecd4-fd2a-4ece-a75d-540d82b7bc71 None None] Registered counter for database keystone Jul 01 03:42:06.718936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG oslo_db.sqlalchemy.engines [None req-5017ecd4-fd2a-4ece-a75d-540d82b7bc71 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80577) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 01 03:42:06.725085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writer thread running {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 01 03:42:06.843035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017ecd4-fd2a-4ece-a75d-540d82b7bc71 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:06.847327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0f477-48fb-4b63-b8c5-228b4e4bf0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:06.847990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f477-48fb-4b63-b8c5-228b4e4bf0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:06.848276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f477-48fb-4b63-b8c5-228b4e4bf0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:06.848552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f477-48fb-4b63-b8c5-228b4e4bf0e1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:06.856722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0f477-48fb-4b63-b8c5-228b4e4bf0e1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:06.908256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d1d0f477-48fb-4b63-b8c5-228b4e4bf0e1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 01 03:42:06.908538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0f477-48fb-4b63-b8c5-228b4e4bf0e1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:06.919854 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Fri Jul 1 03:42:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 03:42:06.925892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:06.940679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208897e-7669-48f8-89a2-1d542dc4085d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:06.945146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d216c-a811-4a7f-ad4d-2e70ac24117e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:06.946262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d216c-a811-4a7f-ad4d-2e70ac24117e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:06.946552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d216c-a811-4a7f-ad4d-2e70ac24117e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:06.946832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d216c-a811-4a7f-ad4d-2e70ac24117e admin admin] PATH_INFO: `/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:06.948873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d216c-a811-4a7f-ad4d-2e70ac24117e admin admin] RBAC: Authorizing `identity:get_project(project_id=e12b5dbdf665489f80381a9cc1ab6bd0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:07.001515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e09d216c-a811-4a7f-ad4d-2e70ac24117e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 01 03:42:07.002083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d216c-a811-4a7f-ad4d-2e70ac24117e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:07.006577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Fri Jul 1 03:42:06 2022] GET /identity/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:09.036607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d44e9-c22c-4888-a38a-11288f7a8d13 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:09.037117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d44e9-c22c-4888-a38a-11288f7a8d13 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:09.037221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d44e9-c22c-4888-a38a-11288f7a8d13 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:09.041310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:09.046702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4acda-f944-41e1-9ca5-50ea7f994576 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:09.047120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4acda-f944-41e1-9ca5-50ea7f994576 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:09.047291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4acda-f944-41e1-9ca5-50ea7f994576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:09.065952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-16a4acda-f944-41e1-9ca5-50ea7f994576 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:09.067529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-16a4acda-f944-41e1-9ca5-50ea7f994576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:09.075984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 01 03:42:09.098951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d44e9-c22c-4888-a38a-11288f7a8d13 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:09.111166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d44e9-c22c-4888-a38a-11288f7a8d13 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:09.115457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98f833-1b92-41d8-8749-dd98e0fa44b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:09.116100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f833-1b92-41d8-8749-dd98e0fa44b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:09.116385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f833-1b92-41d8-8749-dd98e0fa44b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:09.116664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f833-1b92-41d8-8749-dd98e0fa44b4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:09.124551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98f833-1b92-41d8-8749-dd98e0fa44b4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:09.126318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98f833-1b92-41d8-8749-dd98e0fa44b4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Fri Jul 1 03:42:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 03:42:09.141312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4acda-f944-41e1-9ca5-50ea7f994576 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:09.150611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4acda-f944-41e1-9ca5-50ea7f994576 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:09.155668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347008c0-98b8-43f1-8e09-deb2d1f1b79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:09.156538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347008c0-98b8-43f1-8e09-deb2d1f1b79b admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:09.156824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347008c0-98b8-43f1-8e09-deb2d1f1b79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:09.157101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347008c0-98b8-43f1-8e09-deb2d1f1b79b admin admin] PATH_INFO: `/v3/users/d62406877bb84126838f185cae41e3b4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:09.158500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347008c0-98b8-43f1-8e09-deb2d1f1b79b admin admin] RBAC: Authorizing `identity:get_user(user_id=d62406877bb84126838f185cae41e3b4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:09.160388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347008c0-98b8-43f1-8e09-deb2d1f1b79b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:09.161989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Fri Jul 1 03:42:09 2022] GET /identity/v3/users/d62406877bb84126838f185cae41e3b4 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:11.244499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dd6a7-b43e-4284-8c79-2d113da5a1ba None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.245014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dd6a7-b43e-4284-8c79-2d113da5a1ba None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.245086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dd6a7-b43e-4284-8c79-2d113da5a1ba None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:11.246417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:42:11.251722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a70e4-9b62-486b-8ac1-ca34c875ca5e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.252129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a70e4-9b62-486b-8ac1-ca34c875ca5e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.252396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a70e4-9b62-486b-8ac1-ca34c875ca5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:11.269600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-081a70e4-9b62-486b-8ac1-ca34c875ca5e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:11.270793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-081a70e4-9b62-486b-8ac1-ca34c875ca5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:11.305185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352978b-acd9-463d-a5a1-d5bae87fde74 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.305719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352978b-acd9-463d-a5a1-d5bae87fde74 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.306005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352978b-acd9-463d-a5a1-d5bae87fde74 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:11.307600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:11.312798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.313091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.313380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:11.354815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG passlib.handlers.bcrypt [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80577) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 01 03:42:11.355239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:11.355399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG passlib.handlers.bcrypt [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80577) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 01 03:42:11.379778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a70e4-9b62-486b-8ac1-ca34c875ca5e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:11.401197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:11.402889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:11.404584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:11.487087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a70e4-9b62-486b-8ac1-ca34c875ca5e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:11.505291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:11.521860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34f31b-ff72-4181-b587-12d57f65e04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:11.522655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34f31b-ff72-4181-b587-12d57f65e04e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.522940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34f31b-ff72-4181-b587-12d57f65e04e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.523216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34f31b-ff72-4181-b587-12d57f65e04e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:11.528213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:11.538714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34f31b-ff72-4181-b587-12d57f65e04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-6a34f31b-ff72-4181-b587-12d57f65e04e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:11.550669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:11.551558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:11.552387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:11.559126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34f31b-ff72-4181-b587-12d57f65e04e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:11.629392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4135a5-51a6-4768-ab69-466aaf6e9bfb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:11.631752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:11.641297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:11.659408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34f31b-ff72-4181-b587-12d57f65e04e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:11.666319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff579eb-19db-4c14-a40a-a03a0022a648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:11.667245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff579eb-19db-4c14-a40a-a03a0022a648 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.667539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff579eb-19db-4c14-a40a-a03a0022a648 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.667820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff579eb-19db-4c14-a40a-a03a0022a648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:11.701472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:11.702367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.702727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.703040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:11.704022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:11.704372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:11.705918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:11.721870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:11.728470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:11.746618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff579eb-19db-4c14-a40a-a03a0022a648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:11.748183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff579eb-19db-4c14-a40a-a03a0022a648 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:11.832942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b26d65-552b-4f16-a08d-0f480564ab67 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:11.851583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 03:42:11.857610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff579eb-19db-4c14-a40a-a03a0022a648 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:11.860785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:11.868550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54086c97-f0a8-4427-8f1c-dc73b7c98840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:11.869973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54086c97-f0a8-4427-8f1c-dc73b7c98840 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.870254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54086c97-f0a8-4427-8f1c-dc73b7c98840 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.870528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54086c97-f0a8-4427-8f1c-dc73b7c98840 None admin] PATH_INFO: `/v3/users/d62406877bb84126838f185cae41e3b4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:11.871769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54086c97-f0a8-4427-8f1c-dc73b7c98840 None admin] RBAC: Authorizing `identity:get_user(user_id=d62406877bb84126838f185cae41e3b4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:11.873250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54086c97-f0a8-4427-8f1c-dc73b7c98840 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:11.874885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:11 2022] GET /identity/v3/users/d62406877bb84126838f185cae41e3b4 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:11.880370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54086c97-f0a8-4427-8f1c-dc73b7c98840 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:11.896485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:11.959726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff579eb-19db-4c14-a40a-a03a0022a648 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:11.983115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54086c97-f0a8-4427-8f1c-dc73b7c98840 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:11.993330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:11.994175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:11.994506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:11.994790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:12.015941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:12.017423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:12.018218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7186a7-385f-49fd-8e08-397e8715491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:12.019019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7186a7-385f-49fd-8e08-397e8715491f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:12.019331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7186a7-385f-49fd-8e08-397e8715491f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:12.019608 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7186a7-385f-49fd-8e08-397e8715491f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:12.020796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7186a7-385f-49fd-8e08-397e8715491f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:12.022271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7186a7-385f-49fd-8e08-397e8715491f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:12.023921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:12.030242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7186a7-385f-49fd-8e08-397e8715491f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:42:12.039826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:12.040634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:12.041404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:42:12.041404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:42:12.041404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:42:12.041404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:12.041404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:12.047033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:12.104658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:12.113674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:12.132549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7186a7-385f-49fd-8e08-397e8715491f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:12.147764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3efd7a-6808-4981-9f37-44e7da906bc8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:12.167314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc781c8-d057-4317-b73d-91dd15ce7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:12.167974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc781c8-d057-4317-b73d-91dd15ce7687 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:12.168265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc781c8-d057-4317-b73d-91dd15ce7687 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:12.168542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc781c8-d057-4317-b73d-91dd15ce7687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:12.169388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc781c8-d057-4317-b73d-91dd15ce7687 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.domain.id=default, user.id=d62406877bb84126838f185cae41e3b4) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:12.169731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc781c8-d057-4317-b73d-91dd15ce7687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:12.171200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc781c8-d057-4317-b73d-91dd15ce7687 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:12.178361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Fri Jul 1 03:42:12 2022] GET /identity/v3/role_assignments?user.id=d62406877bb84126838f185cae41e3b4&scope.domain.id=default&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:12.184907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76900b1c-1cf5-4c1e-a30c-000f20ae2bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:12.185606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76900b1c-1cf5-4c1e-a30c-000f20ae2bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:12.185946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76900b1c-1cf5-4c1e-a30c-000f20ae2bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:12.186230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76900b1c-1cf5-4c1e-a30c-000f20ae2bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:12.187066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76900b1c-1cf5-4c1e-a30c-000f20ae2bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:12.187446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76900b1c-1cf5-4c1e-a30c-000f20ae2bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:12.188997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76900b1c-1cf5-4c1e-a30c-000f20ae2bb2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:12.211015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:14.734832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff7cc-1376-4aed-845f-c43a5ca2aeb4 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:14.735604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff7cc-1376-4aed-845f-c43a5ca2aeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:14.735604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff7cc-1376-4aed-845f-c43a5ca2aeb4 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:14.736900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:14.748155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8c5c8-764a-4e1f-a7c6-9db01aa28dcc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:14.748909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8c5c8-764a-4e1f-a7c6-9db01aa28dcc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:14.748909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8c5c8-764a-4e1f-a7c6-9db01aa28dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:14.777093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-69c8c5c8-764a-4e1f-a7c6-9db01aa28dcc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:14.783339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-69c8c5c8-764a-4e1f-a7c6-9db01aa28dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:14.882797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931595a-7d0a-4dcd-bcae-e72bf150ea9a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:14.883161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931595a-7d0a-4dcd-bcae-e72bf150ea9a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:14.883443 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931595a-7d0a-4dcd-bcae-e72bf150ea9a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:14.885568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 13/23] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:42:14.896461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 11/24] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:14.898700 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7970-d55d-4f5f-b4ec-e3ada4baae56 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:14.898990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7970-d55d-4f5f-b4ec-e3ada4baae56 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:14.899303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7970-d55d-4f5f-b4ec-e3ada4baae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:14.936353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e06e7970-d55d-4f5f-b4ec-e3ada4baae56 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:14.938092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e06e7970-d55d-4f5f-b4ec-e3ada4baae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:14.954585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8c5c8-764a-4e1f-a7c6-9db01aa28dcc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:15.073139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:15.093893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 14/25] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:15.133393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e7970-d55d-4f5f-b4ec-e3ada4baae56 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:15.193866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8c5c8-764a-4e1f-a7c6-9db01aa28dcc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:15.276626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb29018-8adf-44ec-ad70-1177912bade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:15.278016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29018-8adf-44ec-ad70-1177912bade3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:15.278785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29018-8adf-44ec-ad70-1177912bade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:15.279335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29018-8adf-44ec-ad70-1177912bade3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:15.307987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb29018-8adf-44ec-ad70-1177912bade3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:15.308877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:15.310716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb29018-8adf-44ec-ad70-1177912bade3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:15.356987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e7970-d55d-4f5f-b4ec-e3ada4baae56 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-ceb29018-8adf-44ec-ad70-1177912bade3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:42:15.381048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:15.382611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:15.384142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:42:15.384142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:42:15.384142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:42:15.384142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:15.384142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 12/26] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:15.386276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a39dbf-ce5e-42cb-84b7-a5a3adce1010 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:15.388489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a39dbf-ce5e-42cb-84b7-a5a3adce1010 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:15.388489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a39dbf-ce5e-42cb-84b7-a5a3adce1010 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:15.389818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 13/27] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:15.396803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a39dbf-ce5e-42cb-84b7-a5a3adce1010 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:15.399181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03678c-584f-4038-bb00-8b7a13bb2b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:15.400031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03678c-584f-4038-bb00-8b7a13bb2b3f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:15.400369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03678c-584f-4038-bb00-8b7a13bb2b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:15.400659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03678c-584f-4038-bb00-8b7a13bb2b3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:15.413422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03678c-584f-4038-bb00-8b7a13bb2b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ab03678c-584f-4038-bb00-8b7a13bb2b3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:15.421415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:15.423022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:15.424521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 15/28] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:15.428562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac746f2-ed5a-480f-9fa1-e8c7e3dbf8c6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:15.428833 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac746f2-ed5a-480f-9fa1-e8c7e3dbf8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:15.429076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac746f2-ed5a-480f-9fa1-e8c7e3dbf8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:15.450522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4ac746f2-ed5a-480f-9fa1-e8c7e3dbf8c6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:15.451809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4ac746f2-ed5a-480f-9fa1-e8c7e3dbf8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:15.638430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 16/29] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:15.640497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac746f2-ed5a-480f-9fa1-e8c7e3dbf8c6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:15.735622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:15.748334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a39dbf-ce5e-42cb-84b7-a5a3adce1010 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:15.804319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:15.844650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:15.845643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:15.846141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:15.846594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:15.847568 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:15.848058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:15.849684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:15.877726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 14/30] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:15.878897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:15.881871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac746f2-ed5a-480f-9fa1-e8c7e3dbf8c6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:15.937243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a46fc-7886-441d-85f7-6bb7cf3e3c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:15.938647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a46fc-7886-441d-85f7-6bb7cf3e3c22 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:15.938943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a46fc-7886-441d-85f7-6bb7cf3e3c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:15.939258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a46fc-7886-441d-85f7-6bb7cf3e3c22 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:15.940321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a46fc-7886-441d-85f7-6bb7cf3e3c22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:15.940742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a46fc-7886-441d-85f7-6bb7cf3e3c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:15.943397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a46fc-7886-441d-85f7-6bb7cf3e3c22 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:15.958404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 17/31] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:15.960537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7daf-f95f-49c7-b0d2-197067382ad3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:15.961036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7daf-f95f-49c7-b0d2-197067382ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:15.961516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7daf-f95f-49c7-b0d2-197067382ad3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:15.963105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 18/32] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:15.964294 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf7daf-f95f-49c7-b0d2-197067382ad3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:16.011371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d50a8-e61a-46d7-bd09-67c1484d101c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:16.051674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0708f82-5876-4d50-82b2-72c0d3db8d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:16.053297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0708f82-5876-4d50-82b2-72c0d3db8d10 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.054087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0708f82-5876-4d50-82b2-72c0d3db8d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.054599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0708f82-5876-4d50-82b2-72c0d3db8d10 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.056075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0708f82-5876-4d50-82b2-72c0d3db8d10 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:16.058620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0708f82-5876-4d50-82b2-72c0d3db8d10 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:16.090397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf7daf-f95f-49c7-b0d2-197067382ad3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:16.119736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:16.128717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52da800-70bf-4998-b300-5f78837896c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:16.129388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52da800-70bf-4998-b300-5f78837896c7 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.129675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52da800-70bf-4998-b300-5f78837896c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.129954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52da800-70bf-4998-b300-5f78837896c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.130679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52da800-70bf-4998-b300-5f78837896c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:16.132239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52da800-70bf-4998-b300-5f78837896c7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:16.294135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 15/33] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 1 03:42:15 2022] POST /identity/v3/roles => generated 219 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:16.295529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0708f82-5876-4d50-82b2-72c0d3db8d10 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:16.327584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:16.410274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 19/34] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:42:15 2022] POST /identity/v3/projects => generated 301 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:16.413191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab247b-c980-4ec2-a2c1-5b438b7a44a5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.413713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab247b-c980-4ec2-a2c1-5b438b7a44a5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.414240 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab247b-c980-4ec2-a2c1-5b438b7a44a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.439246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0708f82-5876-4d50-82b2-72c0d3db8d10 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:16.464549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-70ab247b-c980-4ec2-a2c1-5b438b7a44a5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:16.466193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-70ab247b-c980-4ec2-a2c1-5b438b7a44a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:16.513827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd792f-22a1-4569-a9ab-94977f648440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:16.515524 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd792f-22a1-4569-a9ab-94977f648440 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.516044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd792f-22a1-4569-a9ab-94977f648440 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.516563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd792f-22a1-4569-a9ab-94977f648440 None admin] PATH_INFO: `/v3/users/d62406877bb84126838f185cae41e3b4` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.520019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd792f-22a1-4569-a9ab-94977f648440 None admin] RBAC: Authorizing `identity:get_user(user_id=d62406877bb84126838f185cae41e3b4)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:16.527256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd792f-22a1-4569-a9ab-94977f648440 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:16.527445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 16/35] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:16 2022] GET /identity/v3/users/d62406877bb84126838f185cae41e3b4 => generated 247 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:16.530079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e7bd3-5a0c-4634-a074-63e24ce6526a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.530598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e7bd3-5a0c-4634-a074-63e24ce6526a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.531093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e7bd3-5a0c-4634-a074-63e24ce6526a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.533627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 17/36] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:16.534907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e7bd3-5a0c-4634-a074-63e24ce6526a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:16.683063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:16.698478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e7bd3-5a0c-4634-a074-63e24ce6526a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:16.737899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 20/37] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:16.739616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5365ab-0787-41e4-9784-d250cba731ac None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.740057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5365ab-0787-41e4-9784-d250cba731ac None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.740481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5365ab-0787-41e4-9784-d250cba731ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.749081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da648149-d376-4196-8809-c4096328e2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:16.749905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648149-d376-4196-8809-c4096328e2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.750190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648149-d376-4196-8809-c4096328e2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.750472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648149-d376-4196-8809-c4096328e2f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.751840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da648149-d376-4196-8809-c4096328e2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:16.753305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da648149-d376-4196-8809-c4096328e2f7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:16.755300 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 18/38] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:16.761267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ebde-9cc7-4bb1-b348-32ccc859a3ad None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.761557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ebde-9cc7-4bb1-b348-32ccc859a3ad None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.761839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ebde-9cc7-4bb1-b348-32ccc859a3ad None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.763262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 19/39] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:16.763962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6ebde-9cc7-4bb1-b348-32ccc859a3ad None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:16.768716 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ec5365ab-0787-41e4-9784-d250cba731ac None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:16.769956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ec5365ab-0787-41e4-9784-d250cba731ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:16.844589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 21/40] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:16.845974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5365ab-0787-41e4-9784-d250cba731ac None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:16.874730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6ebde-9cc7-4bb1-b348-32ccc859a3ad None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:16.898831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:16.912376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97644dae-3dac-4c10-9c06-c9725ed89d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:16.913262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644dae-3dac-4c10-9c06-c9725ed89d83 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:16.913548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644dae-3dac-4c10-9c06-c9725ed89d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:16.913825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644dae-3dac-4c10-9c06-c9725ed89d83 None admin] PATH_INFO: `/v3/domains/default/users/d62406877bb84126838f185cae41e3b4/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:16.915984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97644dae-3dac-4c10-9c06-c9725ed89d83 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d62406877bb84126838f185cae41e3b4, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:16.917429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97644dae-3dac-4c10-9c06-c9725ed89d83 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:16.963216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5365ab-0787-41e4-9784-d250cba731ac None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:16.997257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd33c3-aeda-48ef-a5e6-99809b5e8b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:17.000222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd33c3-aeda-48ef-a5e6-99809b5e8b2b None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:17.000222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd33c3-aeda-48ef-a5e6-99809b5e8b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:17.000222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd33c3-aeda-48ef-a5e6-99809b5e8b2b None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:17.000222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd33c3-aeda-48ef-a5e6-99809b5e8b2b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:17.000855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd33c3-aeda-48ef-a5e6-99809b5e8b2b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:17.024065 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 20/41] 10.222.0.20 () {58 vars in 1361 bytes} [Fri Jul 1 03:42:16 2022] PUT /identity/v3/domains/default/users/d62406877bb84126838f185cae41e3b4/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:42:17.025358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae00f24-f012-4a5a-9467-cd1798056ad7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:17.025652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae00f24-f012-4a5a-9467-cd1798056ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:17.025931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae00f24-f012-4a5a-9467-cd1798056ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:17.045053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-cae00f24-f012-4a5a-9467-cd1798056ad7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:17.046991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cae00f24-f012-4a5a-9467-cd1798056ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:17.059301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:17.128370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 22/42] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 1 03:42:16 2022] POST /identity/v3/roles => generated 225 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:17.130306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd33c3-aeda-48ef-a5e6-99809b5e8b2b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:17.143042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 21/43] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:17.168724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae00f24-f012-4a5a-9467-cd1798056ad7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:17.275834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:17.281043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd33c3-aeda-48ef-a5e6-99809b5e8b2b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:17.325339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae00f24-f012-4a5a-9467-cd1798056ad7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:17.341182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404cd3f-45e7-4685-b286-525e613cf2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:17.341875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404cd3f-45e7-4685-b286-525e613cf2c8 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:17.342163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404cd3f-45e7-4685-b286-525e613cf2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:17.342439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404cd3f-45e7-4685-b286-525e613cf2c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:17.343260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404cd3f-45e7-4685-b286-525e613cf2c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:17.345185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404cd3f-45e7-4685-b286-525e613cf2c8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:17.370928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92159070-ec62-4b30-91f0-a62c0a501064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:17.371780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92159070-ec62-4b30-91f0-a62c0a501064 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:17.372071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92159070-ec62-4b30-91f0-a62c0a501064 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:17.372355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92159070-ec62-4b30-91f0-a62c0a501064 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:17.373574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92159070-ec62-4b30-91f0-a62c0a501064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:17.375120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92159070-ec62-4b30-91f0-a62c0a501064 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:17.377087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 22/44] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:17.382653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92159070-ec62-4b30-91f0-a62c0a501064 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:17.399182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:17.459556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 23/45] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 1 03:42:17 2022] POST /identity/v3/roles => generated 223 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:17.495087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92159070-ec62-4b30-91f0-a62c0a501064 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:17.656356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743459ca-d223-4d35-b064-f1af5fc31d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:17.657037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743459ca-d223-4d35-b064-f1af5fc31d30 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:17.657328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743459ca-d223-4d35-b064-f1af5fc31d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:17.657614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743459ca-d223-4d35-b064-f1af5fc31d30 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:17.658356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743459ca-d223-4d35-b064-f1af5fc31d30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:17.659875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743459ca-d223-4d35-b064-f1af5fc31d30 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:17.745633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 23/46] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:42:17 2022] POST /identity/v3/projects => generated 312 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:42:19.458727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f815-d2c9-4a7f-b309-12ebf532a0e5 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:19.459303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f815-d2c9-4a7f-b309-12ebf532a0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:19.459759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f815-d2c9-4a7f-b309-12ebf532a0e5 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:19.462253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 24/47] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:19.472084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5d01-1c98-4828-99fe-03644e3935ff None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:19.472599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5d01-1c98-4828-99fe-03644e3935ff None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:19.473094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5d01-1c98-4828-99fe-03644e3935ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:19.500871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-120b5d01-1c98-4828-99fe-03644e3935ff None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:19.502545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-120b5d01-1c98-4828-99fe-03644e3935ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:19.579399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:19.589053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 24/48] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:19.612443 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f815-d2c9-4a7f-b309-12ebf532a0e5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:19.734768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f815-d2c9-4a7f-b309-12ebf532a0e5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:19.774382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecbbc3-152e-4433-8083-4a8e5eaab0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:19.775203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecbbc3-152e-4433-8083-4a8e5eaab0cb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:19.775521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecbbc3-152e-4433-8083-4a8e5eaab0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:19.775798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecbbc3-152e-4433-8083-4a8e5eaab0cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:19.787519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ecbbc3-152e-4433-8083-4a8e5eaab0cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-47ecbbc3-152e-4433-8083-4a8e5eaab0cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:19.795477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:19.797148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:19.798632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 25/49] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:19.803443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b5d01-1c98-4828-99fe-03644e3935ff None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:19.863098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaad84e-dad9-44a9-8ad6-9fbd95e98215 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:19.863437 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaad84e-dad9-44a9-8ad6-9fbd95e98215 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:19.863723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaad84e-dad9-44a9-8ad6-9fbd95e98215 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:19.865161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 26/50] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:42:19.871453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb3f98-6eb9-4026-a616-4a04d11a38c2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:19.871745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb3f98-6eb9-4026-a616-4a04d11a38c2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:19.872075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb3f98-6eb9-4026-a616-4a04d11a38c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:19.890352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4bdb3f98-6eb9-4026-a616-4a04d11a38c2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:19.891641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4bdb3f98-6eb9-4026-a616-4a04d11a38c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:19.907960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:19.920024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b5d01-1c98-4828-99fe-03644e3935ff None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:19.962206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:19.962949 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:19.963282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:19.963555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:19.964339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:19.964667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:19.966223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:19.975647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 27/51] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:19.980991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 25/52] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:19.985964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb3f98-6eb9-4026-a616-4a04d11a38c2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:20.000158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:20.065469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:20.098332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb3f98-6eb9-4026-a616-4a04d11a38c2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:20.120059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e7f4a-fd7c-4d38-9a0c-9587750a76ee None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:20.135457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c4ec-c79b-49ff-b622-9af800ae7ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:20.136356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c4ec-c79b-49ff-b622-9af800ae7ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:20.136642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c4ec-c79b-49ff-b622-9af800ae7ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:20.136917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c4ec-c79b-49ff-b622-9af800ae7ba6 None admin] PATH_INFO: `/v3/users/d62406877bb84126838f185cae41e3b4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:20.138179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c4ec-c79b-49ff-b622-9af800ae7ba6 None admin] RBAC: Authorizing `identity:get_user(user_id=d62406877bb84126838f185cae41e3b4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:20.139707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c4ec-c79b-49ff-b622-9af800ae7ba6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:20.141385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 28/53] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:19 2022] GET /identity/v3/users/d62406877bb84126838f185cae41e3b4 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:20.146843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c4ec-c79b-49ff-b622-9af800ae7ba6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:20.155271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53670a2-4ac3-404e-adcb-6318891a19b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:20.156049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53670a2-4ac3-404e-adcb-6318891a19b6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:20.156333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53670a2-4ac3-404e-adcb-6318891a19b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:20.156610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53670a2-4ac3-404e-adcb-6318891a19b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:20.157862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53670a2-4ac3-404e-adcb-6318891a19b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:20.159403 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53670a2-4ac3-404e-adcb-6318891a19b6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:20.161330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 26/54] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:20.165153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:20.167445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53670a2-4ac3-404e-adcb-6318891a19b6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:20.251245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c4ec-c79b-49ff-b622-9af800ae7ba6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:20.273287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53670a2-4ac3-404e-adcb-6318891a19b6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:20.285869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae013-6608-4751-88b9-679d08adae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:20.286676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae013-6608-4751-88b9-679d08adae6f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:20.286965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae013-6608-4751-88b9-679d08adae6f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:20.287277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae013-6608-4751-88b9-679d08adae6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:20.288584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ae013-6608-4751-88b9-679d08adae6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:20.290077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ae013-6608-4751-88b9-679d08adae6f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:20.292304 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 29/55] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:20.297619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:20.298251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae013-6608-4751-88b9-679d08adae6f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:20.309189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10ebc6-41ce-494e-861a-1e7ba250258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:20.309858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ebc6-41ce-494e-861a-1e7ba250258a None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:20.310144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ebc6-41ce-494e-861a-1e7ba250258a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:20.310427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ebc6-41ce-494e-861a-1e7ba250258a None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:20.311144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10ebc6-41ce-494e-861a-1e7ba250258a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:20.312638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10ebc6-41ce-494e-861a-1e7ba250258a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:20.383970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:20.399478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae013-6608-4751-88b9-679d08adae6f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:20.417807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 27/56] 10.222.0.20 () {60 vars in 1140 bytes} [Fri Jul 1 03:42:20 2022] POST /identity/v3/projects => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:42:20.436899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339e495-85da-4c33-aae4-5311fd6c8b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:20.437591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339e495-85da-4c33-aae4-5311fd6c8b56 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:20.437871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339e495-85da-4c33-aae4-5311fd6c8b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:20.438145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339e495-85da-4c33-aae4-5311fd6c8b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:20.439014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339e495-85da-4c33-aae4-5311fd6c8b56 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.domain.id=default, user.id=d62406877bb84126838f185cae41e3b4) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:20.439375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339e495-85da-4c33-aae4-5311fd6c8b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:20.440902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339e495-85da-4c33-aae4-5311fd6c8b56 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:20.447927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 30/57] 10.222.0.20 () {56 vars in 1322 bytes} [Fri Jul 1 03:42:20 2022] GET /identity/v3/role_assignments?user.id=d62406877bb84126838f185cae41e3b4&scope.domain.id=default&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:22.488859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1e951-c551-4bbb-bba3-bb90b25f9547 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:22.489376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1e951-c551-4bbb-bba3-bb90b25f9547 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:22.489460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1e951-c551-4bbb-bba3-bb90b25f9547 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:22.490777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 28/58] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:22.497071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b65e31-4d61-450b-844b-5a4c89d148a9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:22.497565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b65e31-4d61-450b-844b-5a4c89d148a9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:22.497676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b65e31-4d61-450b-844b-5a4c89d148a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:22.514833 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e6b65e31-4d61-450b-844b-5a4c89d148a9 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:22.516104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e6b65e31-4d61-450b-844b-5a4c89d148a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:22.588806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:22.614620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1e951-c551-4bbb-bba3-bb90b25f9547 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:22.732539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1e951-c551-4bbb-bba3-bb90b25f9547 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:22.767898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881cc77-693a-4860-abe9-44142fecd6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:22.768696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881cc77-693a-4860-abe9-44142fecd6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:22.768989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881cc77-693a-4860-abe9-44142fecd6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:22.769271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881cc77-693a-4860-abe9-44142fecd6e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:22.770492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881cc77-693a-4860-abe9-44142fecd6e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:22.771996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881cc77-693a-4860-abe9-44142fecd6e4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:22.773678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 29/60] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:22.779781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b65e31-4d61-450b-844b-5a4c89d148a9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:22.860889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:22.888654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b65e31-4d61-450b-844b-5a4c89d148a9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:22.927305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29622606-81c1-4ac4-8b12-218a6a7af2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:22.927998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29622606-81c1-4ac4-8b12-218a6a7af2b5 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:22.928289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29622606-81c1-4ac4-8b12-218a6a7af2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:22.928580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29622606-81c1-4ac4-8b12-218a6a7af2b5 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:22.929318 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29622606-81c1-4ac4-8b12-218a6a7af2b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:22.931168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29622606-81c1-4ac4-8b12-218a6a7af2b5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:23.024047 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 32/61] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:42:22 2022] POST /identity/v3/users => generated 275 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:26.215603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae0683-b104-4192-9b97-7691300f3650 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.215603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae0683-b104-4192-9b97-7691300f3650 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.215603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae0683-b104-4192-9b97-7691300f3650 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.215603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 30/62] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:26.226773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1c49c-6932-4371-87a9-497e4f80338f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.227720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1c49c-6932-4371-87a9-497e4f80338f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.228220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1c49c-6932-4371-87a9-497e4f80338f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.248481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e57f3-616b-4ebf-8918-1829ff1fedda None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.249050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e57f3-616b-4ebf-8918-1829ff1fedda None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.249534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e57f3-616b-4ebf-8918-1829ff1fedda None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.251144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 31/63] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:26.254422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-03a1c49c-6932-4371-87a9-497e4f80338f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:26.255711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-03a1c49c-6932-4371-87a9-497e4f80338f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:26.276662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96590547-b4b8-430e-9613-432dc1de5552 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.277303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96590547-b4b8-430e-9613-432dc1de5552 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.277781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96590547-b4b8-430e-9613-432dc1de5552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.302182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-96590547-b4b8-430e-9613-432dc1de5552 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:26.303454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-96590547-b4b8-430e-9613-432dc1de5552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:26.361387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 33/64] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:26.362775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f2d3-389f-47e6-92cd-1b93f6dfcbfa None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.363074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f2d3-389f-47e6-92cd-1b93f6dfcbfa None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.363411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f2d3-389f-47e6-92cd-1b93f6dfcbfa None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.364767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 34/65] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:26.376384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeff6e-6a26-4cad-8111-c7c5f6e7471d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.376951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeff6e-6a26-4cad-8111-c7c5f6e7471d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.377436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeff6e-6a26-4cad-8111-c7c5f6e7471d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.380395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:26.406330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1ebeff6e-6a26-4cad-8111-c7c5f6e7471d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:26.408352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1ebeff6e-6a26-4cad-8111-c7c5f6e7471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:26.421962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 32/66] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:26.423321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96590547-b4b8-430e-9613-432dc1de5552 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:26.494121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:26.503947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 35/67] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:26.505047 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeff6e-6a26-4cad-8111-c7c5f6e7471d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:26.620215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeff6e-6a26-4cad-8111-c7c5f6e7471d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:26.661067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4440d-c89d-4181-accd-0d35d81310ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:26.662174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4440d-c89d-4181-accd-0d35d81310ec None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.662610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4440d-c89d-4181-accd-0d35d81310ec None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.663035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4440d-c89d-4181-accd-0d35d81310ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.676309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4440d-c89d-4181-accd-0d35d81310ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:26.682847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96590547-b4b8-430e-9613-432dc1de5552 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-48c4440d-c89d-4181-accd-0d35d81310ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:26.686212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:26.687322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:26.688181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:26.695172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 36/68] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:42:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:26.696420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4440d-c89d-4181-accd-0d35d81310ec None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:26.757356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c66c9-b8fc-4569-a175-2227708681ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:26.758723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c66c9-b8fc-4569-a175-2227708681ac None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.759867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c66c9-b8fc-4569-a175-2227708681ac None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.760428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c66c9-b8fc-4569-a175-2227708681ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.764774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:26.782106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c66c9-b8fc-4569-a175-2227708681ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4a3c66c9-b8fc-4569-a175-2227708681ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:26.799898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:26.801794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:26.803464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:26.808022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 33/69] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:26.809689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c66c9-b8fc-4569-a175-2227708681ac None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:26.820840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4440d-c89d-4181-accd-0d35d81310ec None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:26.856806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c8b87-067e-4e2f-903c-6264180c21be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:26.857618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c8b87-067e-4e2f-903c-6264180c21be None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.857906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c8b87-067e-4e2f-903c-6264180c21be None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.858191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c8b87-067e-4e2f-903c-6264180c21be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.870477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c8b87-067e-4e2f-903c-6264180c21be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:26.873119 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-e59c8b87-067e-4e2f-903c-6264180c21be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:26.878761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:26.880554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 37/70] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:26.882055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aff9e-34ea-4b95-b3de-a86c49530711 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:26.883430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aff9e-34ea-4b95-b3de-a86c49530711 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:26.883430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aff9e-34ea-4b95-b3de-a86c49530711 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:26.883652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 38/71] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:26.886012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aff9e-34ea-4b95-b3de-a86c49530711 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:26.972419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c66c9-b8fc-4569-a175-2227708681ac None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.012186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.013089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.013636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.014057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.014926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aff9e-34ea-4b95-b3de-a86c49530711 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.015134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:27.015609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:27.017160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:27.027725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 34/72] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:42:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:27.028571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:27.037740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:27.068877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d17d56-46fa-43bb-b9d7-2e0582fca0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.069980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17d56-46fa-43bb-b9d7-2e0582fca0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.070443 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17d56-46fa-43bb-b9d7-2e0582fca0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.070912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17d56-46fa-43bb-b9d7-2e0582fca0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.072744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17d56-46fa-43bb-b9d7-2e0582fca0a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:27.073279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17d56-46fa-43bb-b9d7-2e0582fca0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:27.075633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17d56-46fa-43bb-b9d7-2e0582fca0a0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:27.095525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 39/73] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:27.097582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a511ab-c7da-4922-945b-69c1f5d3ed7c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.098285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a511ab-c7da-4922-945b-69c1f5d3ed7c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.098941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a511ab-c7da-4922-945b-69c1f5d3ed7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.121954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-93a511ab-c7da-4922-945b-69c1f5d3ed7c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:27.123776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-93a511ab-c7da-4922-945b-69c1f5d3ed7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:27.182653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:27.189024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a0bb-77da-4e47-8a32-059671670890 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.231162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 40/74] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:27.233917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.234618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.234910 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.235201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.236053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:27.236364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:27.237954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:27.239086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d68180-6a96-44b7-ab3d-813b28ed38ad None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.239598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d68180-6a96-44b7-ab3d-813b28ed38ad None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.240164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d68180-6a96-44b7-ab3d-813b28ed38ad None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.241987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 41/75] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:27.244295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d68180-6a96-44b7-ab3d-813b28ed38ad None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:27.256522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 35/76] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:27.257635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:27.383670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62567500-7840-4404-bcbb-cc69154df4d1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.385466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d68180-6a96-44b7-ab3d-813b28ed38ad None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.395406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:27.424261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08371-7320-4abb-8408-44a2614dd6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.425149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08371-7320-4abb-8408-44a2614dd6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.425438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08371-7320-4abb-8408-44a2614dd6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.425721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08371-7320-4abb-8408-44a2614dd6f9 None admin] PATH_INFO: `/v3/users/d62406877bb84126838f185cae41e3b4` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.425902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faab5d-5f30-4063-91b3-c8ae5057d9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.426836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faab5d-5f30-4063-91b3-c8ae5057d9be None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.426969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08371-7320-4abb-8408-44a2614dd6f9 None admin] RBAC: Authorizing `identity:get_user(user_id=d62406877bb84126838f185cae41e3b4)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:27.427299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faab5d-5f30-4063-91b3-c8ae5057d9be None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.427648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faab5d-5f30-4063-91b3-c8ae5057d9be None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.428546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08371-7320-4abb-8408-44a2614dd6f9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:27.430233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 36/77] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:27 2022] GET /identity/v3/users/d62406877bb84126838f185cae41e3b4 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:27.431947 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9a962-047c-40a0-96b5-57c904cbab58 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.432255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9a962-047c-40a0-96b5-57c904cbab58 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.432564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9a962-047c-40a0-96b5-57c904cbab58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.445248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faab5d-5f30-4063-91b3-c8ae5057d9be None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:27.446830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faab5d-5f30-4063-91b3-c8ae5057d9be None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:27.448723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 42/78] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:27 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:27.449552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faab5d-5f30-4063-91b3-c8ae5057d9be None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:27.450485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c8e9a962-047c-40a0-96b5-57c904cbab58 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:27.451815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c8e9a962-047c-40a0-96b5-57c904cbab58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:27.475397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:27.528638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 37/79] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:27.531260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9a962-047c-40a0-96b5-57c904cbab58 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:27.563533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faab5d-5f30-4063-91b3-c8ae5057d9be None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.600490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20b453-dd83-4477-b630-227fb568819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.601293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20b453-dd83-4477-b630-227fb568819e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.601576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20b453-dd83-4477-b630-227fb568819e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.601856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20b453-dd83-4477-b630-227fb568819e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.615104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20b453-dd83-4477-b630-227fb568819e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7c20b453-dd83-4477-b630-227fb568819e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:27.623651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:27.624818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 43/80] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:27.626352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20b453-dd83-4477-b630-227fb568819e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:27.657446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9a962-047c-40a0-96b5-57c904cbab58 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.660013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:27.699169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85de0b-ee77-4231-99d0-4ab659efdc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.700068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85de0b-ee77-4231-99d0-4ab659efdc7c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.700352 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85de0b-ee77-4231-99d0-4ab659efdc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.700630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85de0b-ee77-4231-99d0-4ab659efdc7c None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.701852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85de0b-ee77-4231-99d0-4ab659efdc7c None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:27.706776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85de0b-ee77-4231-99d0-4ab659efdc7c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:27.706776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 38/81] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:27 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:27.706776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85de0b-ee77-4231-99d0-4ab659efdc7c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:27.734137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:27.746401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20b453-dd83-4477-b630-227fb568819e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.781935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ec1f-0536-45d1-9c06-60e2e44eadc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.782566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ec1f-0536-45d1-9c06-60e2e44eadc1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.782852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ec1f-0536-45d1-9c06-60e2e44eadc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.783131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ec1f-0536-45d1-9c06-60e2e44eadc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.818461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85de0b-ee77-4231-99d0-4ab659efdc7c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:27.856187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7374de7-abb8-410b-9eb3-58d283405cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:27.856976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374de7-abb8-410b-9eb3-58d283405cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:27.857345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374de7-abb8-410b-9eb3-58d283405cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:27.857501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319ec1f-0536-45d1-9c06-60e2e44eadc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:27.857780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374de7-abb8-410b-9eb3-58d283405cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:27.858984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319ec1f-0536-45d1-9c06-60e2e44eadc1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:27.886793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:27.944346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7374de7-abb8-410b-9eb3-58d283405cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:27.946786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7374de7-abb8-410b-9eb3-58d283405cb7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:27.976452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:27.982138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 44/82] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:27.984583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ec1f-0536-45d1-9c06-60e2e44eadc1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:28.085566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 39/83] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:28.086952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7374de7-abb8-410b-9eb3-58d283405cb7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:28.129525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ec1f-0536-45d1-9c06-60e2e44eadc1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:28.167059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:28.181204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43c241-275f-4334-bb01-0dd4ade845a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:28.182189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43c241-275f-4334-bb01-0dd4ade845a2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:28.182569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43c241-275f-4334-bb01-0dd4ade845a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:28.182947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43c241-275f-4334-bb01-0dd4ade845a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:28.184436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43c241-275f-4334-bb01-0dd4ade845a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:28.186148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43c241-275f-4334-bb01-0dd4ade845a2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:28.188216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 45/84] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:28.189132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43c241-275f-4334-bb01-0dd4ade845a2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:28.238229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7374de7-abb8-410b-9eb3-58d283405cb7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:28.274336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:28.287673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:28.288562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:28.288949 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:28.289323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:28.290723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:28.291217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:28.292772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:28.305893 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 40/85] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:28.307243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:28.329483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43c241-275f-4334-bb01-0dd4ade845a2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:28.375143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6607add-02a4-41b7-abce-66cbafb9ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:28.376162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607add-02a4-41b7-abce-66cbafb9ee5f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:28.376625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607add-02a4-41b7-abce-66cbafb9ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:28.377093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607add-02a4-41b7-abce-66cbafb9ee5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:28.443011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b95193-7a8b-4188-8ff6-fe5df5ed2f6a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:28.459667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:28.471700 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6607add-02a4-41b7-abce-66cbafb9ee5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:28.473940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6607add-02a4-41b7-abce-66cbafb9ee5f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:28.483896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a7d5e-53b5-440a-95e6-4bc7cfb1adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:28.484972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7d5e-53b5-440a-95e6-4bc7cfb1adcc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:28.485394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7d5e-53b5-440a-95e6-4bc7cfb1adcc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:28.485789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7d5e-53b5-440a-95e6-4bc7cfb1adcc None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:28.487257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a7d5e-53b5-440a-95e6-4bc7cfb1adcc None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:28.488905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a7d5e-53b5-440a-95e6-4bc7cfb1adcc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:28.490698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 41/86] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:28 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:28.496715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a7d5e-53b5-440a-95e6-4bc7cfb1adcc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:28.574311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:28.617601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 46/87] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:28.619545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6607add-02a4-41b7-abce-66cbafb9ee5f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:28.644389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a7d5e-53b5-440a-95e6-4bc7cfb1adcc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:28.685767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53473616-edcf-4e90-a29a-7e0ab177d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:28.686792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53473616-edcf-4e90-a29a-7e0ab177d572 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:28.687179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53473616-edcf-4e90-a29a-7e0ab177d572 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:28.687613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53473616-edcf-4e90-a29a-7e0ab177d572 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:28.689138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53473616-edcf-4e90-a29a-7e0ab177d572 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:28.694344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53473616-edcf-4e90-a29a-7e0ab177d572 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:28.694344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 42/88] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:28 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:28.694813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53473616-edcf-4e90-a29a-7e0ab177d572 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:28.741731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:28.753386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6607add-02a4-41b7-abce-66cbafb9ee5f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:28.796232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a9879-6a32-43b0-8dfb-677905224921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:28.797233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a9879-6a32-43b0-8dfb-677905224921 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:28.797625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a9879-6a32-43b0-8dfb-677905224921 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:28.797998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a9879-6a32-43b0-8dfb-677905224921 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:28.799081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a9879-6a32-43b0-8dfb-677905224921 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:28.800910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a9879-6a32-43b0-8dfb-677905224921 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:28.824653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53473616-edcf-4e90-a29a-7e0ab177d572 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:28.849799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:28.861330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e9fa5-1ddd-4eb4-a618-868ca8a0590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:28.862247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9fa5-1ddd-4eb4-a618-868ca8a0590a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:28.862670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9fa5-1ddd-4eb4-a618-868ca8a0590a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:28.862808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9fa5-1ddd-4eb4-a618-868ca8a0590a None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:28.864108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e9fa5-1ddd-4eb4-a618-868ca8a0590a None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:28.865602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e9fa5-1ddd-4eb4-a618-868ca8a0590a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:28.867728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 43/89] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:28 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:28.869078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e9fa5-1ddd-4eb4-a618-868ca8a0590a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:28.875377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 47/90] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:42:28 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:28.877032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a9879-6a32-43b0-8dfb-677905224921 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:29.042851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a9879-6a32-43b0-8dfb-677905224921 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:29.072133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e9fa5-1ddd-4eb4-a618-868ca8a0590a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:29.109583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038436ca-ab15-4c63-93e4-eea4671d5aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:29.110667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038436ca-ab15-4c63-93e4-eea4671d5aaa None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:29.111136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038436ca-ab15-4c63-93e4-eea4671d5aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:29.112695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038436ca-ab15-4c63-93e4-eea4671d5aaa None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:29.113650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038436ca-ab15-4c63-93e4-eea4671d5aaa None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:29.115984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038436ca-ab15-4c63-93e4-eea4671d5aaa None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:29.119111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 48/91] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:28 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:29.120302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038436ca-ab15-4c63-93e4-eea4671d5aaa None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:29.139430 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:29.141694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:29.141694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:29.141694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:29.143779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=e432781ae1bf4d319a43f8749c32be0f, user.id=d62406877bb84126838f185cae41e3b4) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:29.144498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:29.147503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:29.150254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:29.169858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 44/92] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:28 2022] GET /identity/v3/role_assignments?user.id=d62406877bb84126838f185cae41e3b4&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:29.169858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:29.259301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:29.309017 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038436ca-ab15-4c63-93e4-eea4671d5aaa None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:29.358749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655eac5-fc78-4489-876d-198b05c84f86 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:29.389142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:29.392652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:29.393906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:29.395418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:29.396298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] RBAC: Adding query filter params (role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=e432781ae1bf4d319a43f8749c32be0f, user.id=9143316c5cac437da00eea03db182f56) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:29.396623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:29.398151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:29.405750 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 49/93] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:29 2022] GET /identity/v3/role_assignments?user.id=9143316c5cac437da00eea03db182f56&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:29.406644 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:29.421363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a74b3-04ca-4022-8139-d365ee46498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:29.423572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a74b3-04ca-4022-8139-d365ee46498a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:29.423957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a74b3-04ca-4022-8139-d365ee46498a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:29.424214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a74b3-04ca-4022-8139-d365ee46498a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:29.541995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:29.572526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a74b3-04ca-4022-8139-d365ee46498a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:29.575828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a74b3-04ca-4022-8139-d365ee46498a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:29.593281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f50f86-50d8-43bb-9e35-29d76e7762d7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:29.673531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:29.674208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:29.674495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:29.674766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:29.675656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] RBAC: Adding query filter params (role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=e432781ae1bf4d319a43f8749c32be0f, user.id=9143316c5cac437da00eea03db182f56) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:29.675971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:29.677443 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:29.695257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 50/94] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:29 2022] GET /identity/v3/role_assignments?user.id=9143316c5cac437da00eea03db182f56&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=2e43acd667e44e0bbba45926e652a64f => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:29.720497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:29.828878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 45/95] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:29.834642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:30.014731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59467-6c9a-45ec-bef3-81ea6c4b8854 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:30.071931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b119-e4c6-4a00-a1a3-e5ced30244ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:30.073500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b119-e4c6-4a00-a1a3-e5ced30244ed None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:30.073914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b119-e4c6-4a00-a1a3-e5ced30244ed None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:30.074374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b119-e4c6-4a00-a1a3-e5ced30244ed None admin] PATH_INFO: `/v3/projects/286d326be9a34947a5586b9a649b558d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:30.080849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b119-e4c6-4a00-a1a3-e5ced30244ed None admin] RBAC: Authorizing `identity:get_project(project_id=286d326be9a34947a5586b9a649b558d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:30.083670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b119-e4c6-4a00-a1a3-e5ced30244ed None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:30.087501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 51/96] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:29 2022] GET /identity/v3/projects/286d326be9a34947a5586b9a649b558d => generated 312 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:30.095594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a74b3-04ca-4022-8139-d365ee46498a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:30.281923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a74b3-04ca-4022-8139-d365ee46498a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:30.328636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10ee6c-feb2-403a-999f-59c0125f312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:30.329315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10ee6c-feb2-403a-999f-59c0125f312f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:30.329600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10ee6c-feb2-403a-999f-59c0125f312f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:30.329874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10ee6c-feb2-403a-999f-59c0125f312f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:30.330728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10ee6c-feb2-403a-999f-59c0125f312f None admin] RBAC: Adding query filter params (role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=286d326be9a34947a5586b9a649b558d, user.id=9143316c5cac437da00eea03db182f56) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:30.331106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10ee6c-feb2-403a-999f-59c0125f312f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:30.332539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10ee6c-feb2-403a-999f-59c0125f312f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:30.344366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 46/97] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:30 2022] GET /identity/v3/role_assignments?user.id=9143316c5cac437da00eea03db182f56&scope.project.id=286d326be9a34947a5586b9a649b558d&role.id=2e43acd667e44e0bbba45926e652a64f => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:31.971022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b3205-107c-4ddc-9984-5bf82f4af059 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:31.971939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b3205-107c-4ddc-9984-5bf82f4af059 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:31.972398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b3205-107c-4ddc-9984-5bf82f4af059 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:31.974392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 52/98] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:31.981110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc940c1-bf67-4890-805e-545a23951d32 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:31.982086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc940c1-bf67-4890-805e-545a23951d32 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:31.982833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc940c1-bf67-4890-805e-545a23951d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:32.001424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3cc940c1-bf67-4890-805e-545a23951d32 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:32.003029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3cc940c1-bf67-4890-805e-545a23951d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:32.032455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:32.130536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 47/99] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:32.193344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b3205-107c-4ddc-9984-5bf82f4af059 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:32.227347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:32.422679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b3205-107c-4ddc-9984-5bf82f4af059 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:32.497179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e5d50-dd5a-4408-a564-0463e5f65a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:32.498453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e5d50-dd5a-4408-a564-0463e5f65a42 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:32.498966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e5d50-dd5a-4408-a564-0463e5f65a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:32.499501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e5d50-dd5a-4408-a564-0463e5f65a42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:32.521214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e5d50-dd5a-4408-a564-0463e5f65a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7e2e5d50-dd5a-4408-a564-0463e5f65a42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:32.537062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:32.538800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:32.542537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 53/100] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:32.550580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc940c1-bf67-4890-805e-545a23951d32 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:32.759274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc940c1-bf67-4890-805e-545a23951d32 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:32.846419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:32.847614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:32.848127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:32.848609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:32.849923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:32.850479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:32.853779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:32.871448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 48/101] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:32.881465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e5d50-dd5a-4408-a564-0463e5f65a42 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:33.116018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:33.185902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e5d50-dd5a-4408-a564-0463e5f65a42 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:33.254398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae5db-7c95-4c4a-8327-196c8fd33965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:33.256426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae5db-7c95-4c4a-8327-196c8fd33965 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:33.257208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae5db-7c95-4c4a-8327-196c8fd33965 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:33.257997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae5db-7c95-4c4a-8327-196c8fd33965 None admin] PATH_INFO: `/v3/users/d62406877bb84126838f185cae41e3b4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:33.261107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ae5db-7c95-4c4a-8327-196c8fd33965 None admin] RBAC: Authorizing `identity:get_user(user_id=d62406877bb84126838f185cae41e3b4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:33.264202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ae5db-7c95-4c4a-8327-196c8fd33965 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:33.267654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 54/102] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:32 2022] GET /identity/v3/users/d62406877bb84126838f185cae41e3b4 => generated 247 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:33.274418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:33.317313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:33.521284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38508b8d-9b8e-4bff-8add-86f57b35fb4e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:33.597097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e65092-02e1-4bd7-9938-6af827da2172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:33.598575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e65092-02e1-4bd7-9938-6af827da2172 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:33.599125 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e65092-02e1-4bd7-9938-6af827da2172 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:33.599637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e65092-02e1-4bd7-9938-6af827da2172 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:33.604472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e65092-02e1-4bd7-9938-6af827da2172 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:33.607282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e65092-02e1-4bd7-9938-6af827da2172 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:33.610570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 49/103] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:33 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:33.618011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae5db-7c95-4c4a-8327-196c8fd33965 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:33.773059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613dc4b-b79e-4de4-9ed3-e53ebe23991e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:33.773485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613dc4b-b79e-4de4-9ed3-e53ebe23991e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:33.773985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613dc4b-b79e-4de4-9ed3-e53ebe23991e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:33.779586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 50/104] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:33.786077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c390cd-1c29-40bd-8396-c9a15997b3bb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:33.786979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c390cd-1c29-40bd-8396-c9a15997b3bb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:33.791963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c390cd-1c29-40bd-8396-c9a15997b3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:33.849072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d7c390cd-1c29-40bd-8396-c9a15997b3bb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:33.866265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d7c390cd-1c29-40bd-8396-c9a15997b3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:33.872657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae5db-7c95-4c4a-8327-196c8fd33965 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:33.977631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735548f-bbc1-443d-84f9-f4ab4d8ec838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:33.979254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735548f-bbc1-443d-84f9-f4ab4d8ec838 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:33.979764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735548f-bbc1-443d-84f9-f4ab4d8ec838 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:33.980266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735548f-bbc1-443d-84f9-f4ab4d8ec838 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f/users/d62406877bb84126838f185cae41e3b4/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.000833 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735548f-bbc1-443d-84f9-f4ab4d8ec838 None admin] RBAC: Authorizing `identity:create_grant(project_id=e432781ae1bf4d319a43f8749c32be0f, user_id=d62406877bb84126838f185cae41e3b4, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:34.003620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735548f-bbc1-443d-84f9-f4ab4d8ec838 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:34.065876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 51/105] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:34.068380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3d0ff-9999-4c9a-87b5-ae6c37fe4d3d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:34.068892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3d0ff-9999-4c9a-87b5-ae6c37fe4d3d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:34.069385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3d0ff-9999-4c9a-87b5-ae6c37fe4d3d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.072597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 52/106] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:34.085713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd622af6-1ea1-4c03-a44c-aaab143fb5d0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:34.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd622af6-1ea1-4c03-a44c-aaab143fb5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:34.086909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd622af6-1ea1-4c03-a44c-aaab143fb5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.097132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:34.109614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 55/107] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:42:33 2022] PUT /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f/users/d62406877bb84126838f185cae41e3b4/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:42:34.116552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735548f-bbc1-443d-84f9-f4ab4d8ec838 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:34.118767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-cd622af6-1ea1-4c03-a44c-aaab143fb5d0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:34.120923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cd622af6-1ea1-4c03-a44c-aaab143fb5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:34.151655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:34.339100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 53/108] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:34.364359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd622af6-1ea1-4c03-a44c-aaab143fb5d0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:34.524819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735548f-bbc1-443d-84f9-f4ab4d8ec838 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:34.612913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90402281-958e-4563-abc8-9db92c5366dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:34.614802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90402281-958e-4563-abc8-9db92c5366dd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:34.615678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90402281-958e-4563-abc8-9db92c5366dd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:34.616528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90402281-958e-4563-abc8-9db92c5366dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.619998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90402281-958e-4563-abc8-9db92c5366dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:34.623153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90402281-958e-4563-abc8-9db92c5366dd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:34.626653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 56/109] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:34.629272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f090a6-c86c-477e-b9d8-786114add990 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:34.630063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f090a6-c86c-477e-b9d8-786114add990 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:34.630844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f090a6-c86c-477e-b9d8-786114add990 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.633741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 57/110] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:34.635581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f090a6-c86c-477e-b9d8-786114add990 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:34.639204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd622af6-1ea1-4c03-a44c-aaab143fb5d0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:34.726386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b93dd0-0b0e-4554-b176-cea58e791dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:34.727879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93dd0-0b0e-4554-b176-cea58e791dfe None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:34.728385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93dd0-0b0e-4554-b176-cea58e791dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:34.728874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93dd0-0b0e-4554-b176-cea58e791dfe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.743539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:34.762115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b93dd0-0b0e-4554-b176-cea58e791dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-82b93dd0-0b0e-4554-b176-cea58e791dfe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:34.778839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:34.780706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:42:34.782147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:34.785714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 54/111] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:34.788536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8b5d-87e2-4299-8ba3-89d1eafbae89 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:34.791204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8b5d-87e2-4299-8ba3-89d1eafbae89 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:34.791722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8b5d-87e2-4299-8ba3-89d1eafbae89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.793626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:34.822861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-df9b8b5d-87e2-4299-8ba3-89d1eafbae89 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:34.825313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-df9b8b5d-87e2-4299-8ba3-89d1eafbae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:34.837170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f090a6-c86c-477e-b9d8-786114add990 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:34.878515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fd059-26ac-4f9a-b932-3fcd752b18a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:34.879098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd059-26ac-4f9a-b932-3fcd752b18a3 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:34.879596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd059-26ac-4f9a-b932-3fcd752b18a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:34.880069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd059-26ac-4f9a-b932-3fcd752b18a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.881053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fd059-26ac-4f9a-b932-3fcd752b18a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:34.882726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fd059-26ac-4f9a-b932-3fcd752b18a3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:34.956769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 55/112] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:34.959115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0ecd4-ec81-4c37-99ea-c5b14d136251 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:34.959658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0ecd4-ec81-4c37-99ea-c5b14d136251 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:34.960167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0ecd4-ec81-4c37-99ea-c5b14d136251 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:34.962858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 56/113] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:34.964155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0ecd4-ec81-4c37-99ea-c5b14d136251 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:34.997236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 58/114] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:42:34 2022] POST /identity/v3/projects => generated 302 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:34.998574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f600533-b01f-472a-8e5f-39a53de3193b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.000580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f600533-b01f-472a-8e5f-39a53de3193b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.000580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f600533-b01f-472a-8e5f-39a53de3193b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.027085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5f600533-b01f-472a-8e5f-39a53de3193b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:35.028927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5f600533-b01f-472a-8e5f-39a53de3193b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:35.114371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0ecd4-ec81-4c37-99ea-c5b14d136251 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:35.147641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:35.151517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 59/115] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:35.152415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f600533-b01f-472a-8e5f-39a53de3193b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:35.170789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db832a-e286-4c94-84f0-23cf3d2b698a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:35.171517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db832a-e286-4c94-84f0-23cf3d2b698a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.171807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db832a-e286-4c94-84f0-23cf3d2b698a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.172091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db832a-e286-4c94-84f0-23cf3d2b698a None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.172888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db832a-e286-4c94-84f0-23cf3d2b698a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:35.173207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db832a-e286-4c94-84f0-23cf3d2b698a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:35.175307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db832a-e286-4c94-84f0-23cf3d2b698a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:35.189071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 57/116] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:35.190304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b1fa7-d81f-4453-9599-bf09631ee34d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.190601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b1fa7-d81f-4453-9599-bf09631ee34d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.190888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b1fa7-d81f-4453-9599-bf09631ee34d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.192561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 58/117] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:35.193222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b1fa7-d81f-4453-9599-bf09631ee34d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:35.257180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:35.348522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b1fa7-d81f-4453-9599-bf09631ee34d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:35.360052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f600533-b01f-472a-8e5f-39a53de3193b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:35.401685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b1b51-6ad5-48cb-b49a-c200b2fd916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:35.402829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b1b51-6ad5-48cb-b49a-c200b2fd916a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.403677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b1b51-6ad5-48cb-b49a-c200b2fd916a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.404171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b1b51-6ad5-48cb-b49a-c200b2fd916a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.422648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b1b51-6ad5-48cb-b49a-c200b2fd916a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:35.425087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa44403-ed39-4512-84df-8986b10003f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:35.426416 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44403-ed39-4512-84df-8986b10003f5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.426929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44403-ed39-4512-84df-8986b10003f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.427507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44403-ed39-4512-84df-8986b10003f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-890b1b51-6ad5-48cb-b49a-c200b2fd916a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:35.433351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:35.434273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:35.435007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:35.437652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 59/118] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:35.438847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b1b51-6ad5-48cb-b49a-c200b2fd916a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:35.451697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa44403-ed39-4512-84df-8986b10003f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5fa44403-ed39-4512-84df-8986b10003f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:35.462350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:35.463296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:35.464051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:35.465286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 60/119] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:42:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:35.467218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe326f6-7f6c-4803-bb88-6d305759d34e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.467659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe326f6-7f6c-4803-bb88-6d305759d34e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.468062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe326f6-7f6c-4803-bb88-6d305759d34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.491888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fbe326f6-7f6c-4803-bb88-6d305759d34e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:35.495039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fbe326f6-7f6c-4803-bb88-6d305759d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:35.539554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:35.604145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 61/120] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:35.605441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe326f6-7f6c-4803-bb88-6d305759d34e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:35.663071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:35.664106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b1b51-6ad5-48cb-b49a-c200b2fd916a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:35.732679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd076088-8618-4951-b06d-4ef9e00d448e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:35.734287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd076088-8618-4951-b06d-4ef9e00d448e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.734752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd076088-8618-4951-b06d-4ef9e00d448e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.735204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd076088-8618-4951-b06d-4ef9e00d448e None admin] PATH_INFO: `/v3/users/bbeeda964f974965a23f8a5d9dd2be2e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.763686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd076088-8618-4951-b06d-4ef9e00d448e None admin] RBAC: Authorizing `identity:get_user(user_id=bbeeda964f974965a23f8a5d9dd2be2e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:35.765356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe326f6-7f6c-4803-bb88-6d305759d34e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:35.766184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd076088-8618-4951-b06d-4ef9e00d448e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:35.769842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 60/121] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:35 2022] GET /identity/v3/users/bbeeda964f974965a23f8a5d9dd2be2e => generated 289 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:35.775176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd076088-8618-4951-b06d-4ef9e00d448e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:35.818648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:35.820540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.821099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.821598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.823101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:35.823769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:35.826714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:35.844692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 62/122] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:35.846336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:35.925963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd076088-8618-4951-b06d-4ef9e00d448e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:35.972524 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:35.976626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:35.977970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:35.978481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:35.980007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:35.981495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:35.982012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:35.985030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:36.001509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 61/123] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:42:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:36.003200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:36.020827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e926577-ead9-49ac-8fe1-885d1ed4b664 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:36.023986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:36.068741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a337bd-77d2-4d63-9872-b31353407cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:36.070248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a337bd-77d2-4d63-9872-b31353407cba None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:36.070748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a337bd-77d2-4d63-9872-b31353407cba None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:36.071830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a337bd-77d2-4d63-9872-b31353407cba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:36.086000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a337bd-77d2-4d63-9872-b31353407cba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-58a337bd-77d2-4d63-9872-b31353407cba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:36.096637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:36.097546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:36.098382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:36.101777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 63/124] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:36.103218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a337bd-77d2-4d63-9872-b31353407cba None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:36.216562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52c040-cba2-429c-829a-f799e7521e60 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:36.241057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a337bd-77d2-4d63-9872-b31353407cba None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:36.279397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551ac67-72d2-4df2-9ec0-445b8e2b956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:36.280533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ac67-72d2-4df2-9ec0-445b8e2b956c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:36.281047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ac67-72d2-4df2-9ec0-445b8e2b956c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:36.281532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ac67-72d2-4df2-9ec0-445b8e2b956c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:36.289012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25186e69-68c5-45f5-8d4a-eb61eeb96890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:36.289987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25186e69-68c5-45f5-8d4a-eb61eeb96890 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:36.290279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25186e69-68c5-45f5-8d4a-eb61eeb96890 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:36.290564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25186e69-68c5-45f5-8d4a-eb61eeb96890 None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:36.291952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25186e69-68c5-45f5-8d4a-eb61eeb96890 None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:36.293437 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25186e69-68c5-45f5-8d4a-eb61eeb96890 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:36.295373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 64/125] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:36 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:36.295978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25186e69-68c5-45f5-8d4a-eb61eeb96890 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:36.338543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:36.363720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:36.418380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551ac67-72d2-4df2-9ec0-445b8e2b956c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:36.421644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551ac67-72d2-4df2-9ec0-445b8e2b956c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:36.437579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25186e69-68c5-45f5-8d4a-eb61eeb96890 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:36.503244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79ef36-5877-4940-8d41-e2ce3eb42166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:36.504849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79ef36-5877-4940-8d41-e2ce3eb42166 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:36.505420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79ef36-5877-4940-8d41-e2ce3eb42166 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:36.506022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79ef36-5877-4940-8d41-e2ce3eb42166 None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:36.508362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79ef36-5877-4940-8d41-e2ce3eb42166 None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:36.511991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79ef36-5877-4940-8d41-e2ce3eb42166 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:36.515816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 65/126] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:36 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:36.517201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79ef36-5877-4940-8d41-e2ce3eb42166 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:36.562949 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 62/127] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:36.564633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551ac67-72d2-4df2-9ec0-445b8e2b956c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:36.655340 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:36.660851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79ef36-5877-4940-8d41-e2ce3eb42166 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:36.692795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:36.721614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:36.723021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:36.726490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:36.726490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:36.726490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:36.726490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:36.727034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:36.738724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 66/128] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:36.739851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:36.767336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551ac67-72d2-4df2-9ec0-445b8e2b956c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:36.829664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe3cb-ef33-48ee-b1d9-0b69d20b6963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:36.831338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe3cb-ef33-48ee-b1d9-0b69d20b6963 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:36.831834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe3cb-ef33-48ee-b1d9-0b69d20b6963 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:36.832322 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe3cb-ef33-48ee-b1d9-0b69d20b6963 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:36.834865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afe3cb-ef33-48ee-b1d9-0b69d20b6963 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:36.837693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afe3cb-ef33-48ee-b1d9-0b69d20b6963 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:36.841038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 63/129] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:36 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:36.846240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe3cb-ef33-48ee-b1d9-0b69d20b6963 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:36.866542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f05a7-855c-4c3f-a833-66252519c2ff None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:36.905361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55561719-d173-4762-956c-cecf8acbc534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:36.906293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55561719-d173-4762-956c-cecf8acbc534 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:36.906571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55561719-d173-4762-956c-cecf8acbc534 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:36.906809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55561719-d173-4762-956c-cecf8acbc534 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:36.908121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55561719-d173-4762-956c-cecf8acbc534 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:36.909614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55561719-d173-4762-956c-cecf8acbc534 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:36.911484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 67/130] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:36 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:36.912318 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55561719-d173-4762-956c-cecf8acbc534 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:36.962289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:37.002120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe3cb-ef33-48ee-b1d9-0b69d20b6963 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:37.043607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f212c3-e323-4afb-b97c-c3baf0dbc27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:37.044366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f212c3-e323-4afb-b97c-c3baf0dbc27f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.044660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f212c3-e323-4afb-b97c-c3baf0dbc27f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.044927 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f212c3-e323-4afb-b97c-c3baf0dbc27f None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.046358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f212c3-e323-4afb-b97c-c3baf0dbc27f None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:37.047886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f212c3-e323-4afb-b97c-c3baf0dbc27f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:37.049898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 64/131] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:36 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:37.053882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f661251-dfbe-4119-a60b-e466996f6f6b None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.054582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f661251-dfbe-4119-a60b-e466996f6f6b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.055086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f661251-dfbe-4119-a60b-e466996f6f6b None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.058054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 65/132] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:37.060742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f661251-dfbe-4119-a60b-e466996f6f6b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:37.078533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:37.096369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55561719-d173-4762-956c-cecf8acbc534 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:37.151627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b1b59-dcc4-4c41-9f9d-34716504f652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:37.153351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b1b59-dcc4-4c41-9f9d-34716504f652 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.153818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b1b59-dcc4-4c41-9f9d-34716504f652 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.154448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b1b59-dcc4-4c41-9f9d-34716504f652 None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.156939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b1b59-dcc4-4c41-9f9d-34716504f652 None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:37.159530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b1b59-dcc4-4c41-9f9d-34716504f652 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:37.163594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 68/133] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:36 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:37.166282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b1b59-dcc4-4c41-9f9d-34716504f652 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:37.229335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f661251-dfbe-4119-a60b-e466996f6f6b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:37.289069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9f2a8-55c0-43af-b766-5fdd8ad0f61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:37.292040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9f2a8-55c0-43af-b766-5fdd8ad0f61f None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.293120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9f2a8-55c0-43af-b766-5fdd8ad0f61f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.293713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9f2a8-55c0-43af-b766-5fdd8ad0f61f None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f/users/9143316c5cac437da00eea03db182f56/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.297942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9f2a8-55c0-43af-b766-5fdd8ad0f61f None admin] RBAC: Authorizing `identity:create_grant(project_id=e432781ae1bf4d319a43f8749c32be0f, user_id=9143316c5cac437da00eea03db182f56, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:37.300818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9f2a8-55c0-43af-b766-5fdd8ad0f61f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:37.314469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:37.321910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b1b59-dcc4-4c41-9f9d-34716504f652 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:37.365843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18faf6cb-6e2a-4cc0-9387-6bddc5017a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:37.367073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faf6cb-6e2a-4cc0-9387-6bddc5017a1a None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.367566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faf6cb-6e2a-4cc0-9387-6bddc5017a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.367845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faf6cb-6e2a-4cc0-9387-6bddc5017a1a None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f/users/9143316c5cac437da00eea03db182f56/roles/a3df9ed8088248bbae25abcf9937fa11` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.370580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18faf6cb-6e2a-4cc0-9387-6bddc5017a1a None admin] RBAC: Authorizing `identity:create_grant(project_id=e432781ae1bf4d319a43f8749c32be0f, user_id=9143316c5cac437da00eea03db182f56, role_id=a3df9ed8088248bbae25abcf9937fa11)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:37.372188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18faf6cb-6e2a-4cc0-9387-6bddc5017a1a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:37.398857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 66/134] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:42:37 2022] PUT /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f/users/9143316c5cac437da00eea03db182f56/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:42:37.400306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9f2a8-55c0-43af-b766-5fdd8ad0f61f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:37.479681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:37.495036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 69/135] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:42:37 2022] PUT /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f/users/9143316c5cac437da00eea03db182f56/roles/a3df9ed8088248bbae25abcf9937fa11 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:42:37.497257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cebb98-da89-4ddd-a9dd-bad68965458f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.497695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cebb98-da89-4ddd-a9dd-bad68965458f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.498123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cebb98-da89-4ddd-a9dd-bad68965458f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.536846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e3cebb98-da89-4ddd-a9dd-bad68965458f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:37.539643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e3cebb98-da89-4ddd-a9dd-bad68965458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:37.555608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9f2a8-55c0-43af-b766-5fdd8ad0f61f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:37.620957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:37.622621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.623477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.624211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.626081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] RBAC: Adding query filter params (role.id=44ee62ba929b4af4b083efded29d3d5c, scope.project.id=e432781ae1bf4d319a43f8749c32be0f, user.id=bbeeda964f974965a23f8a5d9dd2be2e) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:37.626932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:37.629831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:37.642411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 67/136] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:37 2022] GET /identity/v3/role_assignments?user.id=bbeeda964f974965a23f8a5d9dd2be2e&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=44ee62ba929b4af4b083efded29d3d5c => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:37.649486 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:37.718088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 70/137] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:37.721013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab740f85-a8cc-4688-97af-f5867d731881 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.721581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab740f85-a8cc-4688-97af-f5867d731881 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.722183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab740f85-a8cc-4688-97af-f5867d731881 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.729794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 71/138] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:37.737379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b8c27-759b-4d38-b667-9b079e97e1e0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.738025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b8c27-759b-4d38-b667-9b079e97e1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.738618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b8c27-759b-4d38-b667-9b079e97e1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.777267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-609b8c27-759b-4d38-b667-9b079e97e1e0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:37.779498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-609b8c27-759b-4d38-b667-9b079e97e1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:37.894278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5cd29-d2ee-4be8-948a-5edcf3eb3646 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:37.931192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 72/139] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:37.933045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b8c27-759b-4d38-b667-9b079e97e1e0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:37.937967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:37.946930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8668e-84f7-4b3e-b91b-78c6c8ab5503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:37.948086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8668e-84f7-4b3e-b91b-78c6c8ab5503 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:37.948519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8668e-84f7-4b3e-b91b-78c6c8ab5503 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:37.952171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8668e-84f7-4b3e-b91b-78c6c8ab5503 None admin] PATH_INFO: `/v3/projects/286d326be9a34947a5586b9a649b558d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:37.952171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8668e-84f7-4b3e-b91b-78c6c8ab5503 None admin] RBAC: Authorizing `identity:get_project(project_id=286d326be9a34947a5586b9a649b558d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:37.952747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8668e-84f7-4b3e-b91b-78c6c8ab5503 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:37.954898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 68/140] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:37 2022] GET /identity/v3/projects/286d326be9a34947a5586b9a649b558d => generated 312 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:37.964225 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8668e-84f7-4b3e-b91b-78c6c8ab5503 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:37.975848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:38.156993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b8c27-759b-4d38-b667-9b079e97e1e0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:38.168328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8668e-84f7-4b3e-b91b-78c6c8ab5503 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:38.275905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c34bf-85e3-4bc2-8960-6c35fdebe599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:38.277258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c34bf-85e3-4bc2-8960-6c35fdebe599 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:38.277726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c34bf-85e3-4bc2-8960-6c35fdebe599 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:38.278193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c34bf-85e3-4bc2-8960-6c35fdebe599 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:38.309235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5dda26-24f5-44c6-b5cf-697b4f4a0fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:38.310874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dda26-24f5-44c6-b5cf-697b4f4a0fbb None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:38.311417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dda26-24f5-44c6-b5cf-697b4f4a0fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:38.311932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dda26-24f5-44c6-b5cf-697b4f4a0fbb None admin] PATH_INFO: `/v3/projects/286d326be9a34947a5586b9a649b558d/users/9143316c5cac437da00eea03db182f56/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:38.320178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5dda26-24f5-44c6-b5cf-697b4f4a0fbb None admin] RBAC: Authorizing `identity:create_grant(project_id=286d326be9a34947a5586b9a649b558d, user_id=9143316c5cac437da00eea03db182f56, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:38.320430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c34bf-85e3-4bc2-8960-6c35fdebe599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:38.323310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5dda26-24f5-44c6-b5cf-697b4f4a0fbb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7b0c34bf-85e3-4bc2-8960-6c35fdebe599 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:38.339997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:38.348225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 73/141] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:38.349755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c34bf-85e3-4bc2-8960-6c35fdebe599 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:38.423755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 69/142] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:42:37 2022] PUT /identity/v3/projects/286d326be9a34947a5586b9a649b558d/users/9143316c5cac437da00eea03db182f56/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 03:42:38.425007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5dda26-24f5-44c6-b5cf-697b4f4a0fbb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:38.604014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:38.627953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:38.653827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c34bf-85e3-4bc2-8960-6c35fdebe599 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:38.696931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5dda26-24f5-44c6-b5cf-697b4f4a0fbb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:38.741981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d4e54-2246-457e-8858-10183e19cf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:38.743472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d4e54-2246-457e-8858-10183e19cf78 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:38.743759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d4e54-2246-457e-8858-10183e19cf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:38.744056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d4e54-2246-457e-8858-10183e19cf78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:38.745453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d4e54-2246-457e-8858-10183e19cf78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:38.746972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d4e54-2246-457e-8858-10183e19cf78 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:38.748954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 74/143] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:38.756984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d4e54-2246-457e-8858-10183e19cf78 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:38.800344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:38.801233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:38.801617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:38.801995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:38.802898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:38.803344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:38.804881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:38.823350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 70/144] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:38.835780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:39.018774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d4e54-2246-457e-8858-10183e19cf78 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:39.162424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b6bf6-1258-4ae1-bbdb-5f170951e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:39.163620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b6bf6-1258-4ae1-bbdb-5f170951e832 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:39.164358 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b6bf6-1258-4ae1-bbdb-5f170951e832 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:39.165407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b6bf6-1258-4ae1-bbdb-5f170951e832 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:39.170499 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b6bf6-1258-4ae1-bbdb-5f170951e832 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:39.173301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b6bf6-1258-4ae1-bbdb-5f170951e832 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:39.215553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff869d9-0c5d-41c0-a2af-62c81c3f777a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:39.272613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:39.299073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbe295-5636-4b19-8938-1ab1b08118bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:39.300394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbe295-5636-4b19-8938-1ab1b08118bb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:39.300718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbe295-5636-4b19-8938-1ab1b08118bb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:39.306443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbe295-5636-4b19-8938-1ab1b08118bb None admin] PATH_INFO: `/v3/users/d62406877bb84126838f185cae41e3b4` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:39.308820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbe295-5636-4b19-8938-1ab1b08118bb None admin] RBAC: Authorizing `identity:get_user(user_id=d62406877bb84126838f185cae41e3b4)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:39.311846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbe295-5636-4b19-8938-1ab1b08118bb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:39.315661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 71/145] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:38 2022] GET /identity/v3/users/d62406877bb84126838f185cae41e3b4 => generated 247 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:39.324922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbe295-5636-4b19-8938-1ab1b08118bb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:39.329904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:39.345763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 75/146] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:42:38 2022] POST /identity/v3/users => generated 283 bytes in 589 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:39.595577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbe295-5636-4b19-8938-1ab1b08118bb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:39.677025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09eb62-d062-450e-80fe-c7f8d64abcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:39.678518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09eb62-d062-450e-80fe-c7f8d64abcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:39.679286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09eb62-d062-450e-80fe-c7f8d64abcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:39.679943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09eb62-d062-450e-80fe-c7f8d64abcb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:39.810708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:39.839818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09eb62-d062-450e-80fe-c7f8d64abcb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:39.842752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09eb62-d062-450e-80fe-c7f8d64abcb8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:40.077795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 72/147] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:40.085908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b6bf6-1258-4ae1-bbdb-5f170951e832 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:40.236157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b6bf6-1258-4ae1-bbdb-5f170951e832 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:40.275015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a4cb0-7ca5-4a2f-805c-f14d74d8765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:40.275930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4cb0-7ca5-4a2f-805c-f14d74d8765d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:40.276216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4cb0-7ca5-4a2f-805c-f14d74d8765d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:40.276492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4cb0-7ca5-4a2f-805c-f14d74d8765d None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:40.277792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a4cb0-7ca5-4a2f-805c-f14d74d8765d None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:40.279310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a4cb0-7ca5-4a2f-805c-f14d74d8765d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:40.281356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 76/148] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:40 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:40.287205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09eb62-d062-450e-80fe-c7f8d64abcb8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:40.406831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:40.421843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09eb62-d062-450e-80fe-c7f8d64abcb8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:40.469907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d9343-df6f-4c78-97e1-cc41f27a0951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:40.470581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9343-df6f-4c78-97e1-cc41f27a0951 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:40.470868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9343-df6f-4c78-97e1-cc41f27a0951 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:40.471150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9343-df6f-4c78-97e1-cc41f27a0951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:40.472038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d9343-df6f-4c78-97e1-cc41f27a0951 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=e432781ae1bf4d319a43f8749c32be0f, user.id=d62406877bb84126838f185cae41e3b4) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:40.472356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d9343-df6f-4c78-97e1-cc41f27a0951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:40.473919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d9343-df6f-4c78-97e1-cc41f27a0951 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:40.484682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 73/149] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:40 2022] GET /identity/v3/role_assignments?user.id=d62406877bb84126838f185cae41e3b4&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:40.823748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5dab4-fe9d-429b-8326-0d7b25abafe0 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:40.825300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5dab4-fe9d-429b-8326-0d7b25abafe0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:40.826067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5dab4-fe9d-429b-8326-0d7b25abafe0 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:40.829016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 77/150] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:40.835189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e50dc-4d51-4a0e-845f-51a97a2ff383 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:40.835988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e50dc-4d51-4a0e-845f-51a97a2ff383 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:40.835988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e50dc-4d51-4a0e-845f-51a97a2ff383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:40.860290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0e7e50dc-4d51-4a0e-845f-51a97a2ff383 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:40.861506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0e7e50dc-4d51-4a0e-845f-51a97a2ff383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:40.944000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7bc63-bf80-4af5-bf05-d05b784243f2 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:40.944925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7bc63-bf80-4af5-bf05-d05b784243f2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:40.945746 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7bc63-bf80-4af5-bf05-d05b784243f2 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:40.949732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 78/151] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:40.957279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead66594-f940-4046-9d66-4cc29b3e6de2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:40.958276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead66594-f940-4046-9d66-4cc29b3e6de2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:40.959040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead66594-f940-4046-9d66-4cc29b3e6de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:40.975932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 74/152] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:40.999635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ead66594-f940-4046-9d66-4cc29b3e6de2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:41.002174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ead66594-f940-4046-9d66-4cc29b3e6de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:41.002660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e50dc-4d51-4a0e-845f-51a97a2ff383 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:41.148663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:41.154233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:41.166827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 79/153] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:41.186514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e50dc-4d51-4a0e-845f-51a97a2ff383 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:41.205058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead66594-f940-4046-9d66-4cc29b3e6de2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:41.257384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc4843-37a6-4f71-9661-02f1632eaecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:41.258289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc4843-37a6-4f71-9661-02f1632eaecc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:41.258594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc4843-37a6-4f71-9661-02f1632eaecc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:41.258873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc4843-37a6-4f71-9661-02f1632eaecc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:41.276942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc4843-37a6-4f71-9661-02f1632eaecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-c0fc4843-37a6-4f71-9661-02f1632eaecc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:41.290376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:41.291602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:42:41.292369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:41.295329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 75/154] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:41.301835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc4843-37a6-4f71-9661-02f1632eaecc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:41.507144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead66594-f940-4046-9d66-4cc29b3e6de2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:41.546637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc4843-37a6-4f71-9661-02f1632eaecc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:41.605389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:41.606074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:41.606362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:41.606644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:41.607463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:41.607778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:41.609205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:41.619072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 76/155] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:41.624504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:41.628922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:41.632434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817e8ba-b7dd-4347-bc24-362f0ff84c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:41.633876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817e8ba-b7dd-4347-bc24-362f0ff84c4b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:41.634383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817e8ba-b7dd-4347-bc24-362f0ff84c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:41.634883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817e8ba-b7dd-4347-bc24-362f0ff84c4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:41.689039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817e8ba-b7dd-4347-bc24-362f0ff84c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2817e8ba-b7dd-4347-bc24-362f0ff84c4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:41.731588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:41.732503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:41.733267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:41.733836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 80/156] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 530 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:41.741626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e4a13-4b81-4220-928d-7abbfb4cb343 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:41.744784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817e8ba-b7dd-4347-bc24-362f0ff84c4b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:41.790035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adfd10-bdac-494d-ba12-342432d14378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:41.791004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adfd10-bdac-494d-ba12-342432d14378 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:41.791318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adfd10-bdac-494d-ba12-342432d14378 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:41.791604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adfd10-bdac-494d-ba12-342432d14378 None admin] PATH_INFO: `/v3/users/bbeeda964f974965a23f8a5d9dd2be2e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:41.793027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adfd10-bdac-494d-ba12-342432d14378 None admin] RBAC: Authorizing `identity:get_user(user_id=bbeeda964f974965a23f8a5d9dd2be2e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:41.794530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adfd10-bdac-494d-ba12-342432d14378 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:41.801014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 77/157] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:41 2022] GET /identity/v3/users/bbeeda964f974965a23f8a5d9dd2be2e => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:41.807675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adfd10-bdac-494d-ba12-342432d14378 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:41.955780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:41.973992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:42.039266 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adfd10-bdac-494d-ba12-342432d14378 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:42.120688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817e8ba-b7dd-4347-bc24-362f0ff84c4b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:42.122537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a165706-8e58-4408-aa69-1937c23caadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:42.123505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a165706-8e58-4408-aa69-1937c23caadb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:42.123750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a165706-8e58-4408-aa69-1937c23caadb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:42.124033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a165706-8e58-4408-aa69-1937c23caadb None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:42.125387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a165706-8e58-4408-aa69-1937c23caadb None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:42.126866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a165706-8e58-4408-aa69-1937c23caadb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:42.128652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 78/158] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:41 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:42.129915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf2c9c-ba1f-4316-8dee-8a7ff38b976f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:42.130188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf2c9c-ba1f-4316-8dee-8a7ff38b976f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:42.130467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf2c9c-ba1f-4316-8dee-8a7ff38b976f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:42.132002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 79/159] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:42.136588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf2c9c-ba1f-4316-8dee-8a7ff38b976f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:42.269462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e48b32-f639-4e47-8cb1-d7e2b618e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:42.270561 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e48b32-f639-4e47-8cb1-d7e2b618e121 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:42.271048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e48b32-f639-4e47-8cb1-d7e2b618e121 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:42.271558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e48b32-f639-4e47-8cb1-d7e2b618e121 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:42.272924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e48b32-f639-4e47-8cb1-d7e2b618e121 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:42.273470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e48b32-f639-4e47-8cb1-d7e2b618e121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:42.276565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e48b32-f639-4e47-8cb1-d7e2b618e121 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:42.311651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 81/160] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 568 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:42.313812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fd2ea-12b4-4929-b113-ae2e299f7b98 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:42.314312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fd2ea-12b4-4929-b113-ae2e299f7b98 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:42.314817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fd2ea-12b4-4929-b113-ae2e299f7b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:42.377032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-339fd2ea-12b4-4929-b113-ae2e299f7b98 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:42.378963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-339fd2ea-12b4-4929-b113-ae2e299f7b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:42.380415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf2c9c-ba1f-4316-8dee-8a7ff38b976f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:42.451194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea260e83-caf2-49b9-bc89-6d15df2e65d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:42.452110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260e83-caf2-49b9-bc89-6d15df2e65d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:42.452391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260e83-caf2-49b9-bc89-6d15df2e65d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:42.452660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260e83-caf2-49b9-bc89-6d15df2e65d6 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f/users/bbeeda964f974965a23f8a5d9dd2be2e/roles/44ee62ba929b4af4b083efded29d3d5c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:42.458105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea260e83-caf2-49b9-bc89-6d15df2e65d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e432781ae1bf4d319a43f8749c32be0f, user_id=bbeeda964f974965a23f8a5d9dd2be2e, role_id=44ee62ba929b4af4b083efded29d3d5c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:42.459597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea260e83-caf2-49b9-bc89-6d15df2e65d6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:42.510617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 82/161] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:42.511687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fd2ea-12b4-4929-b113-ae2e299f7b98 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:42.572684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 80/162] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:42:42 2022] PUT /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f/users/bbeeda964f974965a23f8a5d9dd2be2e/roles/44ee62ba929b4af4b083efded29d3d5c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:42:42.574244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e798c1-71b0-434b-8c9e-fdabeb47edf0 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:42.574861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e798c1-71b0-434b-8c9e-fdabeb47edf0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:42.575115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e798c1-71b0-434b-8c9e-fdabeb47edf0 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:42.576651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 81/163] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:42.583134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:42.583982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:42.586842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e798c1-71b0-434b-8c9e-fdabeb47edf0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:42.678121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fd2ea-12b4-4929-b113-ae2e299f7b98 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:42.708977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902c13d-de03-4cd4-9915-0729fa92bb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:42.709867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902c13d-de03-4cd4-9915-0729fa92bb9f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:42.710133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902c13d-de03-4cd4-9915-0729fa92bb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:42.710403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902c13d-de03-4cd4-9915-0729fa92bb9f None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:42.712423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902c13d-de03-4cd4-9915-0729fa92bb9f None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:42.714144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902c13d-de03-4cd4-9915-0729fa92bb9f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:42.716104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 83/164] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:42 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:42.717360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2c31-ecd9-42fc-ab91-beacc316e4c6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:42.717647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2c31-ecd9-42fc-ab91-beacc316e4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:42.718002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2c31-ecd9-42fc-ab91-beacc316e4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:42.737185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9abc2c31-ecd9-42fc-ab91-beacc316e4c6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:42.738466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9abc2c31-ecd9-42fc-ab91-beacc316e4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:42.823867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 84/165] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:42.825021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2c31-ecd9-42fc-ab91-beacc316e4c6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:42.901753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:42.965492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2c31-ecd9-42fc-ab91-beacc316e4c6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:43.004030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88ee88-e0be-490b-8508-77df82297f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:43.004743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88ee88-e0be-490b-8508-77df82297f89 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:43.005026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88ee88-e0be-490b-8508-77df82297f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:43.005295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88ee88-e0be-490b-8508-77df82297f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:43.086018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88ee88-e0be-490b-8508-77df82297f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:43.087584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88ee88-e0be-490b-8508-77df82297f89 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:43.112423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e798c1-71b0-434b-8c9e-fdabeb47edf0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:43.157184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:43.207492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 85/166] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:43.208257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88ee88-e0be-490b-8508-77df82297f89 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:43.264384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367009e9-b898-49e1-a344-a8ff6acd639b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:43.265731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367009e9-b898-49e1-a344-a8ff6acd639b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:43.266222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367009e9-b898-49e1-a344-a8ff6acd639b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:43.266705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367009e9-b898-49e1-a344-a8ff6acd639b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:43.322324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88ee88-e0be-490b-8508-77df82297f89 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:43.324265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367009e9-b898-49e1-a344-a8ff6acd639b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-367009e9-b898-49e1-a344-a8ff6acd639b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:43.344473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:43.345404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:43.346364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:43.346961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 82/167] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:42:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 770 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:43.356277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367009e9-b898-49e1-a344-a8ff6acd639b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:43.361578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad954cc5-d017-41dd-9fd3-9cae2c0c793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:43.362383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad954cc5-d017-41dd-9fd3-9cae2c0c793f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:43.362664 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad954cc5-d017-41dd-9fd3-9cae2c0c793f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:43.362930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad954cc5-d017-41dd-9fd3-9cae2c0c793f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:43.377015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad954cc5-d017-41dd-9fd3-9cae2c0c793f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ad954cc5-d017-41dd-9fd3-9cae2c0c793f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:43.386372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:43.387354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:43.388334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 86/168] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:43.390086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad954cc5-d017-41dd-9fd3-9cae2c0c793f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:43.417778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:43.516859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad954cc5-d017-41dd-9fd3-9cae2c0c793f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:43.518631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:43.563469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:43.564674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:43.565097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:43.565539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:43.567097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:43.567577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:43.569329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:43.579644 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 87/169] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:42:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:43.581592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:43.612114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367009e9-b898-49e1-a344-a8ff6acd639b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:43.671604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:43.709150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac510b68-ac28-47b3-90ab-182062e138dc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:43.747173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb88fc-a7fd-4852-8806-9607e7ba417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:43.749195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb88fc-a7fd-4852-8806-9607e7ba417d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:43.749908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb88fc-a7fd-4852-8806-9607e7ba417d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:43.750607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb88fc-a7fd-4852-8806-9607e7ba417d None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:43.756825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865f6c2-8c16-44c3-99b1-720959031f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:43.757502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865f6c2-8c16-44c3-99b1-720959031f72 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:43.757783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865f6c2-8c16-44c3-99b1-720959031f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:43.758121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865f6c2-8c16-44c3-99b1-720959031f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:43.758891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865f6c2-8c16-44c3-99b1-720959031f72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:43.759298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865f6c2-8c16-44c3-99b1-720959031f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:43.759662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb88fc-a7fd-4852-8806-9607e7ba417d None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:43.760963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865f6c2-8c16-44c3-99b1-720959031f72 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:43.762961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb88fc-a7fd-4852-8806-9607e7ba417d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:43.766690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 83/170] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:43 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:43.775394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 88/171] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:43.775394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4b171-f678-4c00-8601-4adfe22898e2 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:43.775394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4b171-f678-4c00-8601-4adfe22898e2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:43.775394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4b171-f678-4c00-8601-4adfe22898e2 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:43.775394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 89/172] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:43.776749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb88fc-a7fd-4852-8806-9607e7ba417d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:43.778681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4b171-f678-4c00-8601-4adfe22898e2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:43.943090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4b171-f678-4c00-8601-4adfe22898e2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:44.021937 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb88fc-a7fd-4852-8806-9607e7ba417d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:44.024812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:44.050013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee97fb-157c-4bc0-b263-163de7d3845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:44.051606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee97fb-157c-4bc0-b263-163de7d3845e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:44.052408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee97fb-157c-4bc0-b263-163de7d3845e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:44.053172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee97fb-157c-4bc0-b263-163de7d3845e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:44.055053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee97fb-157c-4bc0-b263-163de7d3845e None admin] RBAC: Adding query filter params (role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=e432781ae1bf4d319a43f8749c32be0f, user.id=9143316c5cac437da00eea03db182f56) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:44.055490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:44.056354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee97fb-157c-4bc0-b263-163de7d3845e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:44.067253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9392e-d8f1-4bd6-9f63-91834463042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:44.067253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9392e-d8f1-4bd6-9f63-91834463042e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:44.067253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9392e-d8f1-4bd6-9f63-91834463042e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:44.067253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9392e-d8f1-4bd6-9f63-91834463042e None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:44.067800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9392e-d8f1-4bd6-9f63-91834463042e None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:44.069709 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9392e-d8f1-4bd6-9f63-91834463042e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:44.070294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 84/173] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:43 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:44.070961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9392e-d8f1-4bd6-9f63-91834463042e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:44.071320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee97fb-157c-4bc0-b263-163de7d3845e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:44.091715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 90/174] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:43 2022] GET /identity/v3/role_assignments?user.id=9143316c5cac437da00eea03db182f56&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=2e43acd667e44e0bbba45926e652a64f => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:44.094550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79817ecd-7370-4b33-82a6-2979e1df3927 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:44.095362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79817ecd-7370-4b33-82a6-2979e1df3927 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:44.096093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79817ecd-7370-4b33-82a6-2979e1df3927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:44.140875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-79817ecd-7370-4b33-82a6-2979e1df3927 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:44.142082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-79817ecd-7370-4b33-82a6-2979e1df3927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:44.228665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 91/175] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:44.236345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79817ecd-7370-4b33-82a6-2979e1df3927 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:44.285171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9392e-d8f1-4bd6-9f63-91834463042e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:44.375810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0d5e-a9df-450d-b3cf-1599866fa921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:44.376616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0d5e-a9df-450d-b3cf-1599866fa921 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:44.376891 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0d5e-a9df-450d-b3cf-1599866fa921 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:44.377170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0d5e-a9df-450d-b3cf-1599866fa921 None admin] PATH_INFO: `/v3/users/9143316c5cac437da00eea03db182f56` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:44.380119 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f0d5e-a9df-450d-b3cf-1599866fa921 None admin] RBAC: Authorizing `identity:get_user(user_id=9143316c5cac437da00eea03db182f56)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:44.381612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f0d5e-a9df-450d-b3cf-1599866fa921 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:44.384159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 85/176] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:44 2022] GET /identity/v3/users/9143316c5cac437da00eea03db182f56 => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:44.384993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0d5e-a9df-450d-b3cf-1599866fa921 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:44.426228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79817ecd-7370-4b33-82a6-2979e1df3927 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:44.455628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:44.473095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c7c9-f677-4862-8a5a-f25deebea9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:44.473840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c7c9-f677-4862-8a5a-f25deebea9ec None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:44.475663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c7c9-f677-4862-8a5a-f25deebea9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:44.475663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c7c9-f677-4862-8a5a-f25deebea9ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:44.507820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:44.563574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0d5e-a9df-450d-b3cf-1599866fa921 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:44.605991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbb15e-ca7b-4d18-b274-50320f76ad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:44.606914 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbb15e-ca7b-4d18-b274-50320f76ad2b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:44.607200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbb15e-ca7b-4d18-b274-50320f76ad2b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:44.607517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbb15e-ca7b-4d18-b274-50320f76ad2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:44.622472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbb15e-ca7b-4d18-b274-50320f76ad2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b6fbb15e-ca7b-4d18-b274-50320f76ad2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:44.631422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:44.633987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 86/177] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:44.635612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbb15e-ca7b-4d18-b274-50320f76ad2b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:44.697600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310c7c9-f677-4862-8a5a-f25deebea9ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:44.700838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310c7c9-f677-4862-8a5a-f25deebea9ec None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:44.789295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbb15e-ca7b-4d18-b274-50320f76ad2b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:44.839811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:44.870722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7d9d4-ee62-4a30-829c-118caff7bd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:44.871425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d9d4-ee62-4a30-829c-118caff7bd56 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:44.871705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d9d4-ee62-4a30-829c-118caff7bd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:44.871975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d9d4-ee62-4a30-829c-118caff7bd56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:44.943093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 92/178] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:44.944111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c7c9-f677-4862-8a5a-f25deebea9ec None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:45.006278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:45.018134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7d9d4-ee62-4a30-829c-118caff7bd56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:45.028469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7d9d4-ee62-4a30-829c-118caff7bd56 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:45.091753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c7c9-f677-4862-8a5a-f25deebea9ec None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:45.133518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f4483-5237-44b7-97c4-3eb5632281a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:45.134214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f4483-5237-44b7-97c4-3eb5632281a1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.134492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f4483-5237-44b7-97c4-3eb5632281a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.134760 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f4483-5237-44b7-97c4-3eb5632281a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.135590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f4483-5237-44b7-97c4-3eb5632281a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:45.135892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f4483-5237-44b7-97c4-3eb5632281a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:45.137335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f4483-5237-44b7-97c4-3eb5632281a1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:45.143729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 87/179] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:45.145274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7d9d4-ee62-4a30-829c-118caff7bd56 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:45.147764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 93/180] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:45.149552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a143-6685-4060-aed1-4d60266fd9ab None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.149890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a143-6685-4060-aed1-4d60266fd9ab None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.150130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a143-6685-4060-aed1-4d60266fd9ab None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.151676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 94/181] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:45.156406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2a143-6685-4060-aed1-4d60266fd9ab None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:45.181724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:45.270539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7d9d4-ee62-4a30-829c-118caff7bd56 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:45.272453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:45.289973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2a143-6685-4060-aed1-4d60266fd9ab None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:45.317556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851acbc0-0d99-4f9a-879b-aba28ed92051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:45.318447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851acbc0-0d99-4f9a-879b-aba28ed92051 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.318731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851acbc0-0d99-4f9a-879b-aba28ed92051 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.319006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851acbc0-0d99-4f9a-879b-aba28ed92051 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.320358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851acbc0-0d99-4f9a-879b-aba28ed92051 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:45.321900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851acbc0-0d99-4f9a-879b-aba28ed92051 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:45.328693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 88/182] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:45 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:45.328693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2055-db6b-4ec1-98a5-96ca470d410d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.328693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2055-db6b-4ec1-98a5-96ca470d410d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.328693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2055-db6b-4ec1-98a5-96ca470d410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.344144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b93d2055-db6b-4ec1-98a5-96ca470d410d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:45.345391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b93d2055-db6b-4ec1-98a5-96ca470d410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:45.352543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f12f96-79cb-4451-83a9-c4f4e8a56454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:45.353995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12f96-79cb-4451-83a9-c4f4e8a56454 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.354443 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12f96-79cb-4451-83a9-c4f4e8a56454 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.355412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12f96-79cb-4451-83a9-c4f4e8a56454 None admin] PATH_INFO: `/v3/projects/286d326be9a34947a5586b9a649b558d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.357022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f12f96-79cb-4451-83a9-c4f4e8a56454 None admin] RBAC: Authorizing `identity:get_project(project_id=286d326be9a34947a5586b9a649b558d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:45.359637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f12f96-79cb-4451-83a9-c4f4e8a56454 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:45.362828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 95/183] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:45 2022] GET /identity/v3/projects/286d326be9a34947a5586b9a649b558d => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:45.364103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f12f96-79cb-4451-83a9-c4f4e8a56454 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:45.454617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 89/184] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:45.456408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819fe8b-d9a6-4731-94a5-cc77ca0f6d89 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.456826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819fe8b-d9a6-4731-94a5-cc77ca0f6d89 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.457234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819fe8b-d9a6-4731-94a5-cc77ca0f6d89 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.459021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 90/185] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:45.459941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819fe8b-d9a6-4731-94a5-cc77ca0f6d89 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:45.505987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:45.551515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f12f96-79cb-4451-83a9-c4f4e8a56454 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:45.607955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af499eef-2a0c-49bc-8ac1-11edd7de5a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:45.610252 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af499eef-2a0c-49bc-8ac1-11edd7de5a42 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.610925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af499eef-2a0c-49bc-8ac1-11edd7de5a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.611675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af499eef-2a0c-49bc-8ac1-11edd7de5a42 None admin] PATH_INFO: `/v3/users/f2e5869229bd43e3a95580fef5200ab3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.613032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819fe8b-d9a6-4731-94a5-cc77ca0f6d89 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:45.638407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af499eef-2a0c-49bc-8ac1-11edd7de5a42 None admin] RBAC: Authorizing `identity:get_user(user_id=f2e5869229bd43e3a95580fef5200ab3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:45.640703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af499eef-2a0c-49bc-8ac1-11edd7de5a42 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:45.643217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 96/186] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:45 2022] GET /identity/v3/users/f2e5869229bd43e3a95580fef5200ab3 => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:45.644082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af499eef-2a0c-49bc-8ac1-11edd7de5a42 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:45.659825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1497e0-a087-4727-9518-07e6c1d1023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:45.660838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1497e0-a087-4727-9518-07e6c1d1023a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.661341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1497e0-a087-4727-9518-07e6c1d1023a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.661664 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:45.661781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1497e0-a087-4727-9518-07e6c1d1023a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.662637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1497e0-a087-4727-9518-07e6c1d1023a None admin] RBAC: Adding query filter params (role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=e432781ae1bf4d319a43f8749c32be0f, user.id=9143316c5cac437da00eea03db182f56) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:45.663017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1497e0-a087-4727-9518-07e6c1d1023a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:45.665065 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1497e0-a087-4727-9518-07e6c1d1023a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:45.673814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 91/187] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:45 2022] GET /identity/v3/role_assignments?user.id=9143316c5cac437da00eea03db182f56&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:45.675369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fd911-5ca8-444e-88a8-7006e3a71c05 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.675687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fd911-5ca8-444e-88a8-7006e3a71c05 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.675992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fd911-5ca8-444e-88a8-7006e3a71c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.720733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d84fd911-5ca8-444e-88a8-7006e3a71c05 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:45.722332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d84fd911-5ca8-444e-88a8-7006e3a71c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:45.823528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 92/188] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:45.826126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fd911-5ca8-444e-88a8-7006e3a71c05 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:45.827070 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af499eef-2a0c-49bc-8ac1-11edd7de5a42 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:45.869975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:45.896285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:45.897446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:45.897947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:45.898449 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:45.900234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] RBAC: Adding query filter params (role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=286d326be9a34947a5586b9a649b558d, user.id=9143316c5cac437da00eea03db182f56) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:45.900756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:45.903624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:45.916744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 97/189] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:45 2022] GET /identity/v3/role_assignments?user.id=9143316c5cac437da00eea03db182f56&scope.project.id=286d326be9a34947a5586b9a649b558d&role.id=2e43acd667e44e0bbba45926e652a64f => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:45.918240 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:46.005141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fd911-5ca8-444e-88a8-7006e3a71c05 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:46.050967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:46.059550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4a92d-bf0a-4be1-a555-fa5930e88b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:46.060771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4a92d-bf0a-4be1-a555-fa5930e88b34 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.061262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4a92d-bf0a-4be1-a555-fa5930e88b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.062123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4a92d-bf0a-4be1-a555-fa5930e88b34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.063718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4a92d-bf0a-4be1-a555-fa5930e88b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:46.066079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4a92d-bf0a-4be1-a555-fa5930e88b34 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:46.068763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 93/190] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:46.070480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4a92d-bf0a-4be1-a555-fa5930e88b34 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:46.087026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b78da3-e9e2-4ef1-884f-209776ead6ed None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:46.138622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1ef81-179a-44d9-b08c-eb11a201fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:46.139411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1ef81-179a-44d9-b08c-eb11a201fae0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.139622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1ef81-179a-44d9-b08c-eb11a201fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.140014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1ef81-179a-44d9-b08c-eb11a201fae0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.212694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:46.237249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4a92d-bf0a-4be1-a555-fa5930e88b34 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:46.245466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1ef81-179a-44d9-b08c-eb11a201fae0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:46.247448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1ef81-179a-44d9-b08c-eb11a201fae0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:46.275665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f14e6-06de-4198-9f9a-87105a493d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:46.276819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f14e6-06de-4198-9f9a-87105a493d74 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.277515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f14e6-06de-4198-9f9a-87105a493d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.277515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f14e6-06de-4198-9f9a-87105a493d74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.288713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f14e6-06de-4198-9f9a-87105a493d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-410f14e6-06de-4198-9f9a-87105a493d74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:46.297791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:46.298944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:46.299781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 94/191] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:42:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:46.301092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f14e6-06de-4198-9f9a-87105a493d74 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:46.368360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:46.400200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 98/192] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:46.401496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1ef81-179a-44d9-b08c-eb11a201fae0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:46.459983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f14e6-06de-4198-9f9a-87105a493d74 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:46.504751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255eb8d-1ec6-4ce6-bf19-5a201893a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:46.505598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255eb8d-1ec6-4ce6-bf19-5a201893a9d0 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.505884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255eb8d-1ec6-4ce6-bf19-5a201893a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.506170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255eb8d-1ec6-4ce6-bf19-5a201893a9d0 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.506900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255eb8d-1ec6-4ce6-bf19-5a201893a9d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:46.509000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255eb8d-1ec6-4ce6-bf19-5a201893a9d0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:46.529658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1ef81-179a-44d9-b08c-eb11a201fae0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:46.567589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f2c03-1003-4658-b4bb-eac49281f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:46.568302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f2c03-1003-4658-b4bb-eac49281f5de None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.568578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f2c03-1003-4658-b4bb-eac49281f5de None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.568857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f2c03-1003-4658-b4bb-eac49281f5de None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.569620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f2c03-1003-4658-b4bb-eac49281f5de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:46.569921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f2c03-1003-4658-b4bb-eac49281f5de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:46.571884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f2c03-1003-4658-b4bb-eac49281f5de None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:46.581140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 99/193] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:42:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:46.582563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0decf-5be3-4e27-8ba8-7864a9a027ef None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.582851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0decf-5be3-4e27-8ba8-7864a9a027ef None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.583126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0decf-5be3-4e27-8ba8-7864a9a027ef None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.584604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 100/194] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:46.585172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0decf-5be3-4e27-8ba8-7864a9a027ef None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:46.624072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:46.629166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 95/195] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:42:46 2022] POST /identity/v3/users => generated 297 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:46.629835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255eb8d-1ec6-4ce6-bf19-5a201893a9d0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:46.645242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:46.728476 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0decf-5be3-4e27-8ba8-7864a9a027ef None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:46.772973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc7bbe-9dd0-404c-a78e-b780f227ecb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:46.773936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7bbe-9dd0-404c-a78e-b780f227ecb5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.774219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7bbe-9dd0-404c-a78e-b780f227ecb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.774497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7bbe-9dd0-404c-a78e-b780f227ecb5 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.776012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc7bbe-9dd0-404c-a78e-b780f227ecb5 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:46.777877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc7bbe-9dd0-404c-a78e-b780f227ecb5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:46.780142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 101/196] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:46 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:46.781401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255eb8d-1ec6-4ce6-bf19-5a201893a9d0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:46.786630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3264ed-39f1-48d2-b6cf-e10b068cfd6c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.786913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3264ed-39f1-48d2-b6cf-e10b068cfd6c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.787193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3264ed-39f1-48d2-b6cf-e10b068cfd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.805623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ae3264ed-39f1-48d2-b6cf-e10b068cfd6c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:46.806844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ae3264ed-39f1-48d2-b6cf-e10b068cfd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:46.825298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be0c01-69b7-4db6-812e-b4edec71522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:46.826431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0c01-69b7-4db6-812e-b4edec71522a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:46.826862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0c01-69b7-4db6-812e-b4edec71522a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:46.827298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0c01-69b7-4db6-812e-b4edec71522a None admin] PATH_INFO: `/v3/users/f2e5869229bd43e3a95580fef5200ab3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:46.829627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be0c01-69b7-4db6-812e-b4edec71522a None admin] RBAC: Authorizing `identity:get_user(user_id=f2e5869229bd43e3a95580fef5200ab3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:46.831384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be0c01-69b7-4db6-812e-b4edec71522a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:46.833100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 96/197] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:46 2022] GET /identity/v3/users/f2e5869229bd43e3a95580fef5200ab3 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:46.833884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be0c01-69b7-4db6-812e-b4edec71522a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:46.900321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 102/198] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:46.902611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3264ed-39f1-48d2-b6cf-e10b068cfd6c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:46.970812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:46.972525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:47.042976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be0c01-69b7-4db6-812e-b4edec71522a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:47.092126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3264ed-39f1-48d2-b6cf-e10b068cfd6c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:47.115370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:47.116658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:47.117685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:47.118180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:47.119810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac, user.id=f2e5869229bd43e3a95580fef5200ab3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:47.120510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:47.123800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:47.136336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 97/199] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:46 2022] GET /identity/v3/role_assignments?user.id=f2e5869229bd43e3a95580fef5200ab3&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:47.138233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:47.168559 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dece0b-3c53-4c08-960c-14401ac0b452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:47.170033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dece0b-3c53-4c08-960c-14401ac0b452 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:47.170541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dece0b-3c53-4c08-960c-14401ac0b452 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:47.170920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dece0b-3c53-4c08-960c-14401ac0b452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:47.369617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dece0b-3c53-4c08-960c-14401ac0b452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:47.372048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dece0b-3c53-4c08-960c-14401ac0b452 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:47.433768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68293e-3f76-4792-b187-b437e788c184 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:47.464241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:47.513113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:47.536339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fc3bc-97cd-43dd-a8b6-5cbc7e0f9421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:47.537626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fc3bc-97cd-43dd-a8b6-5cbc7e0f9421 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:47.538079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fc3bc-97cd-43dd-a8b6-5cbc7e0f9421 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:47.538507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fc3bc-97cd-43dd-a8b6-5cbc7e0f9421 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:47.584416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fc3bc-97cd-43dd-a8b6-5cbc7e0f9421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-892fc3bc-97cd-43dd-a8b6-5cbc7e0f9421 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:47.592919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:47.593879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:47.594703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 98/200] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:47.603830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fc3bc-97cd-43dd-a8b6-5cbc7e0f9421 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:47.624249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 103/201] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:47.635684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dece0b-3c53-4c08-960c-14401ac0b452 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:47.826350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fc3bc-97cd-43dd-a8b6-5cbc7e0f9421 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:47.854682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dece0b-3c53-4c08-960c-14401ac0b452 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:47.901539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:47.902230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:47.902515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:47.902792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:47.905393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:47.905708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:47.907110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:47.927496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 99/202] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:47.928807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6069e-01e8-4705-a811-a82ac6c50a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:47.929728 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6069e-01e8-4705-a811-a82ac6c50a69 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:47.930005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6069e-01e8-4705-a811-a82ac6c50a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:47.930527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6069e-01e8-4705-a811-a82ac6c50a69 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:47.931762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6069e-01e8-4705-a811-a82ac6c50a69 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:47.933761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:47.935251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6069e-01e8-4705-a811-a82ac6c50a69 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:47.944123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 104/203] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:47 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:47.946453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6069e-01e8-4705-a811-a82ac6c50a69 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:47.980834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:48.103326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:48.163072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6069e-01e8-4705-a811-a82ac6c50a69 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:48.195898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ecc74-d9a9-44fe-be5e-99c77da1c063 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:48.280658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0793264-dbc8-4cff-9834-8d05d3c618e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:48.282217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793264-dbc8-4cff-9834-8d05d3c618e9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:48.282719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793264-dbc8-4cff-9834-8d05d3c618e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:48.283248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793264-dbc8-4cff-9834-8d05d3c618e9 None admin] PATH_INFO: `/v3/users/bbeeda964f974965a23f8a5d9dd2be2e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:48.286276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0793264-dbc8-4cff-9834-8d05d3c618e9 None admin] RBAC: Authorizing `identity:get_user(user_id=bbeeda964f974965a23f8a5d9dd2be2e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:48.289229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0793264-dbc8-4cff-9834-8d05d3c618e9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:48.292506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 100/204] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:47 2022] GET /identity/v3/users/bbeeda964f974965a23f8a5d9dd2be2e => generated 289 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:48.301724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:48.302397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:48.302674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:48.302944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:48.303829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] RBAC: Adding query filter params (role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac, user.id=f2e5869229bd43e3a95580fef5200ab3) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:48.304142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:48.305757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0793264-dbc8-4cff-9834-8d05d3c618e9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:48.306583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:48.313481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 105/205] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:47 2022] GET /identity/v3/role_assignments?user.id=f2e5869229bd43e3a95580fef5200ab3&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 259 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:48.559276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0793264-dbc8-4cff-9834-8d05d3c618e9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:48.607414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:48.636560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b58db4-082f-473d-afc5-184e572a65af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:48.637649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b58db4-082f-473d-afc5-184e572a65af None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:48.638129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b58db4-082f-473d-afc5-184e572a65af None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:48.638615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b58db4-082f-473d-afc5-184e572a65af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:48.799041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b58db4-082f-473d-afc5-184e572a65af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:48.801767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b58db4-082f-473d-afc5-184e572a65af None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:49.012124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 101/206] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:42:49.020381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:49.132550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:49.208595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787faf2-2b0e-483a-9b5a-8f05706e0eab None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:49.302977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a7582-e214-43f7-85bd-5a51f621dacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:49.304560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7582-e214-43f7-85bd-5a51f621dacd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:49.305290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7582-e214-43f7-85bd-5a51f621dacd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:49.305800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7582-e214-43f7-85bd-5a51f621dacd None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:49.312856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a7582-e214-43f7-85bd-5a51f621dacd None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:49.315761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a7582-e214-43f7-85bd-5a51f621dacd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:49.319194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 106/207] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:49 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:49.329076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b58db4-082f-473d-afc5-184e572a65af None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:49.395813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:49.549708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e72192-c5dd-4129-9a75-41d2acd73065 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:49.550168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e72192-c5dd-4129-9a75-41d2acd73065 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:49.550624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e72192-c5dd-4129-9a75-41d2acd73065 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:49.553256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 107/208] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:49.565647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa935a-426d-4bbf-84f4-abb4ac18529d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:49.566098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa935a-426d-4bbf-84f4-abb4ac18529d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:49.566785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa935a-426d-4bbf-84f4-abb4ac18529d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:49.598084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f2fa935a-426d-4bbf-84f4-abb4ac18529d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:49.600796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f2fa935a-426d-4bbf-84f4-abb4ac18529d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:49.601806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b58db4-082f-473d-afc5-184e572a65af None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:49.752376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da62436-8c47-4e54-93ee-19e7d283754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:49.753493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62436-8c47-4e54-93ee-19e7d283754a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:49.753988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62436-8c47-4e54-93ee-19e7d283754a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:49.754471 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62436-8c47-4e54-93ee-19e7d283754a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:49.756064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da62436-8c47-4e54-93ee-19e7d283754a None admin] RBAC: Adding query filter params (role.id=44ee62ba929b4af4b083efded29d3d5c, scope.project.id=e432781ae1bf4d319a43f8749c32be0f, user.id=bbeeda964f974965a23f8a5d9dd2be2e) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:49.756595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da62436-8c47-4e54-93ee-19e7d283754a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:49.758463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 108/209] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:49.763211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da62436-8c47-4e54-93ee-19e7d283754a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:49.783399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa935a-426d-4bbf-84f4-abb4ac18529d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:49.783993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 102/210] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:49 2022] GET /identity/v3/role_assignments?user.id=bbeeda964f974965a23f8a5d9dd2be2e&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=44ee62ba929b4af4b083efded29d3d5c => generated 608 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:49.941259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:50.008727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa935a-426d-4bbf-84f4-abb4ac18529d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:50.031501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c851e8-ecbe-40d8-b87d-d1851938ee90 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:50.032379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c851e8-ecbe-40d8-b87d-d1851938ee90 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:50.033164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c851e8-ecbe-40d8-b87d-d1851938ee90 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:50.036071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 103/211] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:50.042303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c85052-f697-4e3e-931d-e5d21837050d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:50.042926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c85052-f697-4e3e-931d-e5d21837050d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:50.043420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c85052-f697-4e3e-931d-e5d21837050d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:50.064732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-59c85052-f697-4e3e-931d-e5d21837050d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:50.066306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-59c85052-f697-4e3e-931d-e5d21837050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:50.078956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8927ea-dba0-4151-8dde-b5e1234af1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:50.080192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8927ea-dba0-4151-8dde-b5e1234af1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:50.080694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8927ea-dba0-4151-8dde-b5e1234af1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:50.081263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8927ea-dba0-4151-8dde-b5e1234af1e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:50.082774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8927ea-dba0-4151-8dde-b5e1234af1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:50.084636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8927ea-dba0-4151-8dde-b5e1234af1e2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:50.090144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 109/212] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:50.093396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8927ea-dba0-4151-8dde-b5e1234af1e2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:50.191993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 104/213] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:50.217227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c85052-f697-4e3e-931d-e5d21837050d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:50.237511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:50.292818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8927ea-dba0-4151-8dde-b5e1234af1e2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:50.332165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c85052-f697-4e3e-931d-e5d21837050d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:50.366657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892c3f5-a51e-487b-a6ae-91a8d3ad0f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:50.368656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892c3f5-a51e-487b-a6ae-91a8d3ad0f65 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:50.369310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892c3f5-a51e-487b-a6ae-91a8d3ad0f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:50.369950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892c3f5-a51e-487b-a6ae-91a8d3ad0f65 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:50.371426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892c3f5-a51e-487b-a6ae-91a8d3ad0f65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:50.374279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892c3f5-a51e-487b-a6ae-91a8d3ad0f65 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:50.387254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:50.396542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c10cc8-4490-4d95-9deb-5ae667124ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:50.397546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c10cc8-4490-4d95-9deb-5ae667124ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:50.397932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c10cc8-4490-4d95-9deb-5ae667124ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:50.403646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c10cc8-4490-4d95-9deb-5ae667124ed4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:50.413026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c10cc8-4490-4d95-9deb-5ae667124ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-54c10cc8-4490-4d95-9deb-5ae667124ed4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:50.421434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:50.423746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:50.425454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 105/214] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:50.431082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c10cc8-4490-4d95-9deb-5ae667124ed4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:50.512914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 110/215] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:42:50 2022] POST /identity/v3/users => generated 297 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:50.565704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:50.596246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c10cc8-4490-4d95-9deb-5ae667124ed4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:50.638310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:50.639902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:50.640473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:50.640957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:50.641949 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:50.642471 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:50.644232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:50.655617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 106/216] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:50.664823 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892c3f5-a51e-487b-a6ae-91a8d3ad0f65 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:50.861419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892c3f5-a51e-487b-a6ae-91a8d3ad0f65 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:50.936264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95414477-550c-440d-9216-5ab3a4e41678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:50.937682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95414477-550c-440d-9216-5ab3a4e41678 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:50.938150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95414477-550c-440d-9216-5ab3a4e41678 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:50.938632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95414477-550c-440d-9216-5ab3a4e41678 None admin] PATH_INFO: `/v3/users/23bbb254af924001856937b007559ca9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:50.970018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95414477-550c-440d-9216-5ab3a4e41678 None admin] RBAC: Authorizing `identity:get_user(user_id=23bbb254af924001856937b007559ca9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:50.973468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95414477-550c-440d-9216-5ab3a4e41678 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:50.978119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 111/217] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:50 2022] GET /identity/v3/users/23bbb254af924001856937b007559ca9 => generated 297 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:50.986328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:51.049869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d07e8-7c9b-480b-90a3-cfa98816c7d8 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:51.050817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d07e8-7c9b-480b-90a3-cfa98816c7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:51.051627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d07e8-7c9b-480b-90a3-cfa98816c7d8 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:51.055526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 112/218] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:42:51.067634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104113e9-9258-45b3-a13b-ab58f5183d88 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:51.068570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104113e9-9258-45b3-a13b-ab58f5183d88 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:51.069325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104113e9-9258-45b3-a13b-ab58f5183d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:51.121840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-104113e9-9258-45b3-a13b-ab58f5183d88 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:51.124665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-104113e9-9258-45b3-a13b-ab58f5183d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:51.257931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:51.261420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34b55a-a4f3-4183-8fb1-c749a4fa8dcc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:51.300996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4e445-1b5d-485a-aa36-120e3354ffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:51.301717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4e445-1b5d-485a-aa36-120e3354ffcb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:51.302001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4e445-1b5d-485a-aa36-120e3354ffcb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:51.302417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4e445-1b5d-485a-aa36-120e3354ffcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:51.320154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:51.325193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 113/219] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:51.372311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104113e9-9258-45b3-a13b-ab58f5183d88 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:51.415264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4e445-1b5d-485a-aa36-120e3354ffcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:51.415264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4e445-1b5d-485a-aa36-120e3354ffcb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:51.539450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 107/220] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 03:42:51.545374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4e445-1b5d-485a-aa36-120e3354ffcb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:51.611275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:51.681058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4e445-1b5d-485a-aa36-120e3354ffcb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:51.697674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104113e9-9258-45b3-a13b-ab58f5183d88 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:51.752639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ac81-54d5-4753-a16b-8f0c931c68a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:51.753523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ac81-54d5-4753-a16b-8f0c931c68a4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:51.753806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ac81-54d5-4753-a16b-8f0c931c68a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:51.754084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ac81-54d5-4753-a16b-8f0c931c68a4 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:51.755412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286ac81-54d5-4753-a16b-8f0c931c68a4 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:51.756892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286ac81-54d5-4753-a16b-8f0c931c68a4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:51.760637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 108/221] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:51 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:51.769344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262f74f-d067-41aa-8021-75feae224e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:51.770157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262f74f-d067-41aa-8021-75feae224e2a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:51.770436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262f74f-d067-41aa-8021-75feae224e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:51.770711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262f74f-d067-41aa-8021-75feae224e2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:51.772190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ac81-54d5-4753-a16b-8f0c931c68a4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:51.802456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262f74f-d067-41aa-8021-75feae224e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d262f74f-d067-41aa-8021-75feae224e2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:51.810669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:51.811620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:51.812399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 114/222] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:51.824332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262f74f-d067-41aa-8021-75feae224e2a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:51.960583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:52.010281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ac81-54d5-4753-a16b-8f0c931c68a4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:52.029572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:52.061039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262f74f-d067-41aa-8021-75feae224e2a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:52.092720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8205f-c272-4d5f-93bb-60f40f951975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:52.093421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8205f-c272-4d5f-93bb-60f40f951975 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:52.093711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8205f-c272-4d5f-93bb-60f40f951975 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:52.093989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8205f-c272-4d5f-93bb-60f40f951975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:52.095649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8205f-c272-4d5f-93bb-60f40f951975 None admin] RBAC: Adding query filter params (role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac, user.id=23bbb254af924001856937b007559ca9) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:52.095973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8205f-c272-4d5f-93bb-60f40f951975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:52.097857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8205f-c272-4d5f-93bb-60f40f951975 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:52.106054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 109/223] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:51 2022] GET /identity/v3/role_assignments?user.id=23bbb254af924001856937b007559ca9&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=2e43acd667e44e0bbba45926e652a64f => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:52.108155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4d7f-1a9a-4487-9306-c044c0f0f277 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:52.108447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4d7f-1a9a-4487-9306-c044c0f0f277 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:52.108732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4d7f-1a9a-4487-9306-c044c0f0f277 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:52.110147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 110/224] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:52.118357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960cd2c-9485-4d44-8b82-f847eb4fe943 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:52.118702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960cd2c-9485-4d44-8b82-f847eb4fe943 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:52.123270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960cd2c-9485-4d44-8b82-f847eb4fe943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:52.129005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:52.130787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:52.131073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:52.131394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:52.132237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:52.133166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:52.134687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:52.155036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 115/225] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:52.159261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e960cd2c-9485-4d44-8b82-f847eb4fe943 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:52.159261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e960cd2c-9485-4d44-8b82-f847eb4fe943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:52.160711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:52.308062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586ec5e-8490-4dcd-b811-a966108d795a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:52.310384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:52.343936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 111/226] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:52.365700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960cd2c-9485-4d44-8b82-f847eb4fe943 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:52.413474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca92b55-45a7-4dec-a5ff-2bc9be39c821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:52.415368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca92b55-45a7-4dec-a5ff-2bc9be39c821 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:52.415666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca92b55-45a7-4dec-a5ff-2bc9be39c821 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:52.416005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca92b55-45a7-4dec-a5ff-2bc9be39c821 None admin] PATH_INFO: `/v3/users/f2e5869229bd43e3a95580fef5200ab3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:52.418607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca92b55-45a7-4dec-a5ff-2bc9be39c821 None admin] RBAC: Authorizing `identity:get_user(user_id=f2e5869229bd43e3a95580fef5200ab3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:52.422850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca92b55-45a7-4dec-a5ff-2bc9be39c821 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:52.423871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 116/227] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:52 2022] GET /identity/v3/users/f2e5869229bd43e3a95580fef5200ab3 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:52.431731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca92b55-45a7-4dec-a5ff-2bc9be39c821 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:52.564691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:52.569679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960cd2c-9485-4d44-8b82-f847eb4fe943 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:52.634185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca92b55-45a7-4dec-a5ff-2bc9be39c821 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:52.654251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c1983-d5ee-471f-846b-a3dbba486d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:52.655087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c1983-d5ee-471f-846b-a3dbba486d03 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:52.655407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c1983-d5ee-471f-846b-a3dbba486d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:52.655685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c1983-d5ee-471f-846b-a3dbba486d03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:52.668433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c1983-d5ee-471f-846b-a3dbba486d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d23c1983-d5ee-471f-846b-a3dbba486d03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:52.677117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:52.678068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:52.679314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 112/228] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:42:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:52.685150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c1983-d5ee-471f-846b-a3dbba486d03 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:52.731189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebb50d-d37a-450e-973a-eae85a55fed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:52.732101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebb50d-d37a-450e-973a-eae85a55fed4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:52.732380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebb50d-d37a-450e-973a-eae85a55fed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:52.732653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebb50d-d37a-450e-973a-eae85a55fed4 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:52.733945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebb50d-d37a-450e-973a-eae85a55fed4 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:52.735484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebb50d-d37a-450e-973a-eae85a55fed4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:52.737628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 117/229] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:52 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:52.749027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebb50d-d37a-450e-973a-eae85a55fed4 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:52.813017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c1983-d5ee-471f-846b-a3dbba486d03 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:52.818669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:52.868559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:52.873932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:52.873932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:52.873932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:52.873932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:52.873932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:52.873932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:52.890972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 113/230] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:42:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:52.897410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:52.951848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:52.953334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebb50d-d37a-450e-973a-eae85a55fed4 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:53.022312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69923f08-9f00-4b48-92b9-1c73f7fb0f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:53.023593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923f08-9f00-4b48-92b9-1c73f7fb0f34 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.024063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923f08-9f00-4b48-92b9-1c73f7fb0f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.024547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923f08-9f00-4b48-92b9-1c73f7fb0f34 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/users/f2e5869229bd43e3a95580fef5200ab3/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.028119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69923f08-9f00-4b48-92b9-1c73f7fb0f34 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc9f58284ad54c079cfbd7d29dbf50ac, user_id=f2e5869229bd43e3a95580fef5200ab3, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:53.029873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69923f08-9f00-4b48-92b9-1c73f7fb0f34 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:53.074892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8ecec-e67f-4e2c-959b-f03196a926fe None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:53.115457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 118/231] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:42:52 2022] PUT /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/users/f2e5869229bd43e3a95580fef5200ab3/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:42:53.126622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a9c92-a4d7-4026-88ee-c680391bd4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:53.127603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a9c92-a4d7-4026-88ee-c680391bd4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.128003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a9c92-a4d7-4026-88ee-c680391bd4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.128288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a9c92-a4d7-4026-88ee-c680391bd4d6 None admin] PATH_INFO: `/v3/users/f2e5869229bd43e3a95580fef5200ab3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.132291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a9c92-a4d7-4026-88ee-c680391bd4d6 None admin] RBAC: Authorizing `identity:get_user(user_id=f2e5869229bd43e3a95580fef5200ab3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:53.134590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a9c92-a4d7-4026-88ee-c680391bd4d6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:53.136552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 114/232] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:52 2022] GET /identity/v3/users/f2e5869229bd43e3a95580fef5200ab3 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:53.142822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69923f08-9f00-4b48-92b9-1c73f7fb0f34 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:53.237200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e95d-9a2e-4b9b-94cb-3c1d34200893 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.237557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e95d-9a2e-4b9b-94cb-3c1d34200893 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.237760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e95d-9a2e-4b9b-94cb-3c1d34200893 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.239192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 115/233] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:53.254224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c631ce-5b6f-49d2-8f48-886feb963007 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.254540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c631ce-5b6f-49d2-8f48-886feb963007 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.254798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c631ce-5b6f-49d2-8f48-886feb963007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.284820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d2c631ce-5b6f-49d2-8f48-886feb963007 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:53.287780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d2c631ce-5b6f-49d2-8f48-886feb963007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:53.320602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:53.328480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69923f08-9f00-4b48-92b9-1c73f7fb0f34 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:53.393763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324e6d-75c6-4972-b173-b8f6485439f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:53.394777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324e6d-75c6-4972-b173-b8f6485439f1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.395058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324e6d-75c6-4972-b173-b8f6485439f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.395367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324e6d-75c6-4972-b173-b8f6485439f1 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.396691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd324e6d-75c6-4972-b173-b8f6485439f1 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:53.398219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd324e6d-75c6-4972-b173-b8f6485439f1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:53.401213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 119/234] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:53 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:53.408780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324e6d-75c6-4972-b173-b8f6485439f1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:53.426747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 116/235] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:53.455917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c631ce-5b6f-49d2-8f48-886feb963007 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:53.488768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:53.564996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324e6d-75c6-4972-b173-b8f6485439f1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:53.631626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa99824-0938-408d-95e5-a325ed8b2717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:53.633510 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa99824-0938-408d-95e5-a325ed8b2717 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.634227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa99824-0938-408d-95e5-a325ed8b2717 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.634931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa99824-0938-408d-95e5-a325ed8b2717 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/users/f2e5869229bd43e3a95580fef5200ab3/roles/a3df9ed8088248bbae25abcf9937fa11` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.639165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa99824-0938-408d-95e5-a325ed8b2717 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc9f58284ad54c079cfbd7d29dbf50ac, user_id=f2e5869229bd43e3a95580fef5200ab3, role_id=a3df9ed8088248bbae25abcf9937fa11)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:53.642335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa99824-0938-408d-95e5-a325ed8b2717 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:53.645655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c631ce-5b6f-49d2-8f48-886feb963007 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:53.738781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da02a67-6ccf-4f71-9da0-fb92468236b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:53.741395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02a67-6ccf-4f71-9da0-fb92468236b8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.742437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02a67-6ccf-4f71-9da0-fb92468236b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.743150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02a67-6ccf-4f71-9da0-fb92468236b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.743450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 120/236] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:42:53 2022] PUT /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/users/f2e5869229bd43e3a95580fef5200ab3/roles/a3df9ed8088248bbae25abcf9937fa11 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 03:42:53.745816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da02a67-6ccf-4f71-9da0-fb92468236b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:53.746039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c5055-0d57-49a5-868f-0c20fea8de83 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.746538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c5055-0d57-49a5-868f-0c20fea8de83 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.749626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da02a67-6ccf-4f71-9da0-fb92468236b8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:53.752090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c5055-0d57-49a5-868f-0c20fea8de83 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.754552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 117/237] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:53.754829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 121/238] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:53.763166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c5055-0d57-49a5-868f-0c20fea8de83 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:53.771373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9fb1a-3625-42af-9f81-a1efc0fd1efd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:53.772262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9fb1a-3625-42af-9f81-a1efc0fd1efd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:53.772898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9fb1a-3625-42af-9f81-a1efc0fd1efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:53.807804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4fe9fb1a-3625-42af-9f81-a1efc0fd1efd None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:53.810320 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4fe9fb1a-3625-42af-9f81-a1efc0fd1efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:53.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:54.066595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 118/239] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:54.116670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9fb1a-3625-42af-9f81-a1efc0fd1efd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:54.126234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c5055-0d57-49a5-868f-0c20fea8de83 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:54.162313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:54.223258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ddc33-4ec1-4202-9382-dc6a4fd58302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:54.223258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ddc33-4ec1-4202-9382-dc6a4fd58302 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:54.223258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ddc33-4ec1-4202-9382-dc6a4fd58302 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:54.223258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ddc33-4ec1-4202-9382-dc6a4fd58302 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:54.223874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ddc33-4ec1-4202-9382-dc6a4fd58302 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:54.224005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ddc33-4ec1-4202-9382-dc6a4fd58302 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:54.339913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9fb1a-3625-42af-9f81-a1efc0fd1efd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:54.368984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 122/240] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:42:53 2022] POST /identity/v3/users => generated 293 bytes in 607 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:42:54.411921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c713-6f61-4c5b-b74f-200d13b28272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:54.414091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c713-6f61-4c5b-b74f-200d13b28272 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:54.414816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c713-6f61-4c5b-b74f-200d13b28272 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:54.415614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c713-6f61-4c5b-b74f-200d13b28272 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:54.437739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c713-6f61-4c5b-b74f-200d13b28272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-96b3c713-6f61-4c5b-b74f-200d13b28272 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:54.454267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:54.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:42:54.457609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:54.462348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 119/241] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:54.472334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ddc33-4ec1-4202-9382-dc6a4fd58302 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:54.681682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:54.717895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ddc33-4ec1-4202-9382-dc6a4fd58302 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:54.761705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cb559-d5f1-4711-9870-e6a32c339360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:54.762403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cb559-d5f1-4711-9870-e6a32c339360 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:54.762703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cb559-d5f1-4711-9870-e6a32c339360 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:54.762967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cb559-d5f1-4711-9870-e6a32c339360 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:54.763778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cb559-d5f1-4711-9870-e6a32c339360 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:54.764088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cb559-d5f1-4711-9870-e6a32c339360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:54.765546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cb559-d5f1-4711-9870-e6a32c339360 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:54.775952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 123/242] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:54.781301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c713-6f61-4c5b-b74f-200d13b28272 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:54.884238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:54.909744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c713-6f61-4c5b-b74f-200d13b28272 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:54.971057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d6d6-d675-43bf-bf6b-860d3a75ce47 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:54.971298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d6d6-d675-43bf-bf6b-860d3a75ce47 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:54.971568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d6d6-d675-43bf-bf6b-860d3a75ce47 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:54.973336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 124/243] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:54.979288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24690c6d-51cb-4af3-9d74-37a255ece72e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:54.979682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24690c6d-51cb-4af3-9d74-37a255ece72e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:54.979907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24690c6d-51cb-4af3-9d74-37a255ece72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:54.999377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-24690c6d-51cb-4af3-9d74-37a255ece72e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:55.000652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-24690c6d-51cb-4af3-9d74-37a255ece72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:55.033148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0bef2-a700-4b8e-b0b2-81a1794bebcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:55.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0bef2-a700-4b8e-b0b2-81a1794bebcb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:55.034339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0bef2-a700-4b8e-b0b2-81a1794bebcb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:55.034617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0bef2-a700-4b8e-b0b2-81a1794bebcb None admin] PATH_INFO: `/v3/users/4162ebb0570646d799b1421fc5c1ada2` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:55.059051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0bef2-a700-4b8e-b0b2-81a1794bebcb None admin] RBAC: Authorizing `identity:get_user(user_id=4162ebb0570646d799b1421fc5c1ada2)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:55.060668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0bef2-a700-4b8e-b0b2-81a1794bebcb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:55.062561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 120/244] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:54 2022] GET /identity/v3/users/4162ebb0570646d799b1421fc5c1ada2 => generated 297 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:55.068281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0bef2-a700-4b8e-b0b2-81a1794bebcb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:55.087374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 125/245] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:42:55.107139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24690c6d-51cb-4af3-9d74-37a255ece72e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:55.197079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0bef2-a700-4b8e-b0b2-81a1794bebcb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:55.199026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:55.234978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24690c6d-51cb-4af3-9d74-37a255ece72e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:55.246527 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4440c1-2460-4d25-b7b1-5d87728565a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:55.247346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4440c1-2460-4d25-b7b1-5d87728565a2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:55.247634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4440c1-2460-4d25-b7b1-5d87728565a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:55.247904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4440c1-2460-4d25-b7b1-5d87728565a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:55.252025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:55.275084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b8506-1e60-42ff-851c-b254ec7c9ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:55.275925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8506-1e60-42ff-851c-b254ec7c9ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:55.276206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8506-1e60-42ff-851c-b254ec7c9ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:55.276495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8506-1e60-42ff-851c-b254ec7c9ef6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:55.287903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b8506-1e60-42ff-851c-b254ec7c9ef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-6d1b8506-1e60-42ff-851c-b254ec7c9ef6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:55.295756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:55.299405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:55.302511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 126/246] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:55.312261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b8506-1e60-42ff-851c-b254ec7c9ef6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:55.336716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4440c1-2460-4d25-b7b1-5d87728565a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:55.338569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4440c1-2460-4d25-b7b1-5d87728565a2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:55.413392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b8506-1e60-42ff-851c-b254ec7c9ef6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:55.450697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:55.451427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:55.451712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:55.451986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:55.452819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:55.453137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:55.454597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:55.456309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:55.464979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 127/247] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:42:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:55.470838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:55.508269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 121/248] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 03:42:55.514362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4440c1-2460-4d25-b7b1-5d87728565a2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:55.576809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:55.580781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e60478-3c1b-4852-bafa-f4deaa2a353e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:55.619171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8db92-a36c-49d1-bcf8-5be51986eb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:55.620108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8db92-a36c-49d1-bcf8-5be51986eb43 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:55.620392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8db92-a36c-49d1-bcf8-5be51986eb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:55.620667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8db92-a36c-49d1-bcf8-5be51986eb43 None admin] PATH_INFO: `/v3/users/23bbb254af924001856937b007559ca9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:55.621956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8db92-a36c-49d1-bcf8-5be51986eb43 None admin] RBAC: Authorizing `identity:get_user(user_id=23bbb254af924001856937b007559ca9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:55.623474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8db92-a36c-49d1-bcf8-5be51986eb43 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:55.629896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 128/249] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:55 2022] GET /identity/v3/users/23bbb254af924001856937b007559ca9 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:55.631310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8db92-a36c-49d1-bcf8-5be51986eb43 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:55.707542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4440c1-2460-4d25-b7b1-5d87728565a2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:55.707887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:55.781707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8db92-a36c-49d1-bcf8-5be51986eb43 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:55.797531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a064-2719-438a-96a7-609521c0141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:55.798461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a064-2719-438a-96a7-609521c0141b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:55.798717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a064-2719-438a-96a7-609521c0141b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:55.798980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a064-2719-438a-96a7-609521c0141b None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:55.804188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660a064-2719-438a-96a7-609521c0141b None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:55.806088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660a064-2719-438a-96a7-609521c0141b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:55.807983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 122/250] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:55 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:55.813982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a064-2719-438a-96a7-609521c0141b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:55.829654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8273e-333e-48bf-a542-95016e262386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:55.830536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8273e-333e-48bf-a542-95016e262386 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:55.830814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8273e-333e-48bf-a542-95016e262386 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:55.831199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8273e-333e-48bf-a542-95016e262386 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:55.833183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8273e-333e-48bf-a542-95016e262386 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:55.834681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8273e-333e-48bf-a542-95016e262386 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:55.847170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 129/251] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:55 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:55.860836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8273e-333e-48bf-a542-95016e262386 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:56.018337 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a064-2719-438a-96a7-609521c0141b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:56.027416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:56.070662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61a830-9db0-4def-8e75-fe6ce9cc87c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:56.071110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61a830-9db0-4def-8e75-fe6ce9cc87c5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:56.071422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61a830-9db0-4def-8e75-fe6ce9cc87c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:56.075254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61a830-9db0-4def-8e75-fe6ce9cc87c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:56.075254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61a830-9db0-4def-8e75-fe6ce9cc87c5 None admin] RBAC: Adding query filter params (role.id=44ee62ba929b4af4b083efded29d3d5c, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac, user.id=4162ebb0570646d799b1421fc5c1ada2) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:56.075254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61a830-9db0-4def-8e75-fe6ce9cc87c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:56.075254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61a830-9db0-4def-8e75-fe6ce9cc87c5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:56.089362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 123/252] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:55 2022] GET /identity/v3/role_assignments?user.id=4162ebb0570646d799b1421fc5c1ada2&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=44ee62ba929b4af4b083efded29d3d5c => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:56.110203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8273e-333e-48bf-a542-95016e262386 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:56.147739 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:56.159779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369797a-539d-49c8-a9b5-d50271a1f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:56.160848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369797a-539d-49c8-a9b5-d50271a1f9d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:56.161133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369797a-539d-49c8-a9b5-d50271a1f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:56.161409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369797a-539d-49c8-a9b5-d50271a1f9d5 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/users/23bbb254af924001856937b007559ca9/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:56.164326 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369797a-539d-49c8-a9b5-d50271a1f9d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc9f58284ad54c079cfbd7d29dbf50ac, user_id=23bbb254af924001856937b007559ca9, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:56.165923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369797a-539d-49c8-a9b5-d50271a1f9d5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:56.298076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 130/253] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:42:55 2022] PUT /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/users/23bbb254af924001856937b007559ca9/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 03:42:56.750747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb910a-2750-4ea8-af3c-eee90e15bfa7 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:56.751244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb910a-2750-4ea8-af3c-eee90e15bfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:56.751739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb910a-2750-4ea8-af3c-eee90e15bfa7 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:56.754304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 124/254] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:42:56.761226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d944c69-dead-4220-90b9-c4eba61de637 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:56.761740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d944c69-dead-4220-90b9-c4eba61de637 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:56.762201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d944c69-dead-4220-90b9-c4eba61de637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:56.797040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1d944c69-dead-4220-90b9-c4eba61de637 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:56.798967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1d944c69-dead-4220-90b9-c4eba61de637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:56.809370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa5c4-1f15-4546-9daf-d910909a8748 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:56.809897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa5c4-1f15-4546-9daf-d910909a8748 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:56.810265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa5c4-1f15-4546-9daf-d910909a8748 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:56.818261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 125/255] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:56 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:56.825204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2495c1-1e86-4595-9a9e-09aae643bafb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:56.825683 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2495c1-1e86-4595-9a9e-09aae643bafb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:56.826134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2495c1-1e86-4595-9a9e-09aae643bafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:56.860963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fe2495c1-1e86-4595-9a9e-09aae643bafb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:56.863150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fe2495c1-1e86-4595-9a9e-09aae643bafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:57.010459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 131/256] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:57.047875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d944c69-dead-4220-90b9-c4eba61de637 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:57.065726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 126/257] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:57.150516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2495c1-1e86-4595-9a9e-09aae643bafb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:57.228718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d944c69-dead-4220-90b9-c4eba61de637 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:57.294116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:57.328904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e98dc-ca28-47c9-bb40-2a18225eda2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:57.330197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e98dc-ca28-47c9-bb40-2a18225eda2c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:57.330713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e98dc-ca28-47c9-bb40-2a18225eda2c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:57.331207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e98dc-ca28-47c9-bb40-2a18225eda2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:57.351487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e98dc-ca28-47c9-bb40-2a18225eda2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-744e98dc-ca28-47c9-bb40-2a18225eda2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:57.359800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:57.360759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:42:57.361549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:42:57.369635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 132/258] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:42:57.380700 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e98dc-ca28-47c9-bb40-2a18225eda2c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:57.381311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:57.386388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2495c1-1e86-4595-9a9e-09aae643bafb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:57.501494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6821268-ebe5-4d26-9085-2b4567b9a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:57.502321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6821268-ebe5-4d26-9085-2b4567b9a05e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:57.502612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6821268-ebe5-4d26-9085-2b4567b9a05e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:57.502892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6821268-ebe5-4d26-9085-2b4567b9a05e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:57.543943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6821268-ebe5-4d26-9085-2b4567b9a05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-c6821268-ebe5-4d26-9085-2b4567b9a05e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:42:57.552288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:42:57.553189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:42:57.553955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:42:57.559740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 127/259] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:42:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:42:57.565557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6821268-ebe5-4d26-9085-2b4567b9a05e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:57.629680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e98dc-ca28-47c9-bb40-2a18225eda2c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:57.729253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:57.730360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:57.730860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:57.731411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:57.732778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:57.733298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:57.735870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:57.752434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 133/260] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:57.761001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:57.883803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:57.917708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6821268-ebe5-4d26-9085-2b4567b9a05e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:58.009602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e64695-9362-49e4-a4f4-491b98afefd5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:58.012505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:58.013640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:58.014125 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:58.014574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:58.015925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:58.016469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:58.019026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:58.048430 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 128/261] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:42:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:58.057888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:58.119411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:58.121740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0230a0-22f6-4f71-aa5e-53708d8657f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:58.123005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0230a0-22f6-4f71-aa5e-53708d8657f0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:58.123467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0230a0-22f6-4f71-aa5e-53708d8657f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:58.123897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0230a0-22f6-4f71-aa5e-53708d8657f0 None admin] PATH_INFO: `/v3/users/f2e5869229bd43e3a95580fef5200ab3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:58.128029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0230a0-22f6-4f71-aa5e-53708d8657f0 None admin] RBAC: Authorizing `identity:get_user(user_id=f2e5869229bd43e3a95580fef5200ab3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:58.129797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0230a0-22f6-4f71-aa5e-53708d8657f0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:58.131965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 134/262] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:57 2022] GET /identity/v3/users/f2e5869229bd43e3a95580fef5200ab3 => generated 283 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:58.139144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0230a0-22f6-4f71-aa5e-53708d8657f0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:58.388090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6a7c1-5dc1-499d-b6ed-2b12d42b2c66 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:58.421676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0230a0-22f6-4f71-aa5e-53708d8657f0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:58.448067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:58.492079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a079e-b7e1-4f66-a083-08fe6218d0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:58.493097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a079e-b7e1-4f66-a083-08fe6218d0ad None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:58.493508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a079e-b7e1-4f66-a083-08fe6218d0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:58.493784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a079e-b7e1-4f66-a083-08fe6218d0ad None admin] PATH_INFO: `/v3/users/f2e5869229bd43e3a95580fef5200ab3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:58.495151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a079e-b7e1-4f66-a083-08fe6218d0ad None admin] RBAC: Authorizing `identity:get_user(user_id=f2e5869229bd43e3a95580fef5200ab3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:58.496653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a079e-b7e1-4f66-a083-08fe6218d0ad None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:58.497554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab786714-5bd6-4f07-b185-c66cb07c7850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:58.498390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786714-5bd6-4f07-b185-c66cb07c7850 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:58.498678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 129/263] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:42:58 2022] GET /identity/v3/users/f2e5869229bd43e3a95580fef5200ab3 => generated 283 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:42:58.499027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786714-5bd6-4f07-b185-c66cb07c7850 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:58.499485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786714-5bd6-4f07-b185-c66cb07c7850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:58.504193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a079e-b7e1-4f66-a083-08fe6218d0ad None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:58.681620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab786714-5bd6-4f07-b185-c66cb07c7850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:58.684611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab786714-5bd6-4f07-b185-c66cb07c7850 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:58.783666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:58.837402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a079e-b7e1-4f66-a083-08fe6218d0ad None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:58.964140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85379927-68c6-4882-8e7d-83e45935a5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:58.964760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85379927-68c6-4882-8e7d-83e45935a5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:58.965002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85379927-68c6-4882-8e7d-83e45935a5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:58.965464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85379927-68c6-4882-8e7d-83e45935a5c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:58.967175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 135/264] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 830 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 03:42:58.977044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab786714-5bd6-4f07-b185-c66cb07c7850 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:59.017596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:59.156483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85379927-68c6-4882-8e7d-83e45935a5c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:59.157987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85379927-68c6-4882-8e7d-83e45935a5c6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:59.212725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab786714-5bd6-4f07-b185-c66cb07c7850 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:59.277173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576e095-94cc-47c9-9db7-606ee09c73fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:59.279175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e095-94cc-47c9-9db7-606ee09c73fa None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:59.279986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e095-94cc-47c9-9db7-606ee09c73fa None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:59.280724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e095-94cc-47c9-9db7-606ee09c73fa None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:59.283136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576e095-94cc-47c9-9db7-606ee09c73fa None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:59.286045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576e095-94cc-47c9-9db7-606ee09c73fa None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:59.290771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 136/265] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:58 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:59.293475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea4a46-086f-416d-af6d-b417e79ec326 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:59.294291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea4a46-086f-416d-af6d-b417e79ec326 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:59.295022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea4a46-086f-416d-af6d-b417e79ec326 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:59.297788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 137/266] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:59.299451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:59.300615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3281d-f22b-4aab-bc65-9eae814e2fcc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:59.301389 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3281d-f22b-4aab-bc65-9eae814e2fcc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:59.302167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3281d-f22b-4aab-bc65-9eae814e2fcc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:59.305949 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 138/267] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:42:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:42:59.307951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3281d-f22b-4aab-bc65-9eae814e2fcc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:59.319614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 130/268] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:42:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 816 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 03:42:59.320993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3b666-019c-4eca-853c-e3cf483091b4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:59.321310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3b666-019c-4eca-853c-e3cf483091b4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:59.321642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3b666-019c-4eca-853c-e3cf483091b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:59.349787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-57f3b666-019c-4eca-853c-e3cf483091b4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:59.351582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-57f3b666-019c-4eca-853c-e3cf483091b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:59.437068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:59.462022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3281d-f22b-4aab-bc65-9eae814e2fcc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:59.486231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 131/269] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:59.489165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86327566-298a-42cc-b6cc-83fbf96b46b9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:59.489983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86327566-298a-42cc-b6cc-83fbf96b46b9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:59.491033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86327566-298a-42cc-b6cc-83fbf96b46b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:59.504334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:59.504986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:59.505270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:59.505548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:59.506400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac, user.id=f2e5869229bd43e3a95580fef5200ab3) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:42:59.506796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:59.508275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:59.515015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 139/270] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:59 2022] GET /identity/v3/role_assignments?user.id=f2e5869229bd43e3a95580fef5200ab3&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:59.515905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:59.530227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-86327566-298a-42cc-b6cc-83fbf96b46b9 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:42:59.533078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-86327566-298a-42cc-b6cc-83fbf96b46b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:42:59.649516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7f3bc-4909-4759-a5f0-4b7eafe3bec1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:59.708762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b8c55-4c39-4110-8733-4007d02727cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:59.712858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b8c55-4c39-4110-8733-4007d02727cf None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:59.714064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b8c55-4c39-4110-8733-4007d02727cf None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:59.715171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b8c55-4c39-4110-8733-4007d02727cf None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:59.719735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b8c55-4c39-4110-8733-4007d02727cf None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:59.724847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b8c55-4c39-4110-8733-4007d02727cf None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:59.735160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 140/271] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:42:59 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:59.736749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b8c55-4c39-4110-8733-4007d02727cf None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:59.748946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 132/272] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:42:59.749842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86327566-298a-42cc-b6cc-83fbf96b46b9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:59.786071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:59.868790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:42:59.887888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b8c55-4c39-4110-8733-4007d02727cf None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:42:59.928675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912ffe7-79c1-4637-84a2-1e19e0de3967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:42:59.929526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912ffe7-79c1-4637-84a2-1e19e0de3967 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:42:59.929838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912ffe7-79c1-4637-84a2-1e19e0de3967 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:42:59.930401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912ffe7-79c1-4637-84a2-1e19e0de3967 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:42:59.932952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912ffe7-79c1-4637-84a2-1e19e0de3967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:42:59.934071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912ffe7-79c1-4637-84a2-1e19e0de3967 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:42:59.936158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 141/273] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:42:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:42:59.936993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912ffe7-79c1-4637-84a2-1e19e0de3967 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:42:59.973078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86327566-298a-42cc-b6cc-83fbf96b46b9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:00.031942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5168468-30c8-41a8-a399-31573bf79a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:00.032787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5168468-30c8-41a8-a399-31573bf79a28 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.033263 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5168468-30c8-41a8-a399-31573bf79a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.033710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5168468-30c8-41a8-a399-31573bf79a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.034664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5168468-30c8-41a8-a399-31573bf79a28 None admin] RBAC: Adding query filter params (role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac, user.id=f2e5869229bd43e3a95580fef5200ab3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:00.035029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5168468-30c8-41a8-a399-31573bf79a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:00.037038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5168468-30c8-41a8-a399-31573bf79a28 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:00.039397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:00.044976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 133/274] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:42:59 2022] GET /identity/v3/role_assignments?user.id=f2e5869229bd43e3a95580fef5200ab3&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:00.047209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c3959-6b04-4f52-b999-2a99db33135e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.047740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c3959-6b04-4f52-b999-2a99db33135e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.048223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c3959-6b04-4f52-b999-2a99db33135e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.051046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 134/275] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:00.052258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c3959-6b04-4f52-b999-2a99db33135e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:00.081645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912ffe7-79c1-4637-84a2-1e19e0de3967 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:00.130861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f444c4-9cab-47de-84e6-eac64d907c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:00.131729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f444c4-9cab-47de-84e6-eac64d907c3e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.132016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f444c4-9cab-47de-84e6-eac64d907c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.132813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f444c4-9cab-47de-84e6-eac64d907c3e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.148428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f444c4-9cab-47de-84e6-eac64d907c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-10f444c4-9cab-47de-84e6-eac64d907c3e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:00.156377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:00.158064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 142/276] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:42:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:00.159556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620252-f152-4f0c-9dcc-8d50c9669320 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.160877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620252-f152-4f0c-9dcc-8d50c9669320 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.160877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620252-f152-4f0c-9dcc-8d50c9669320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.179275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e8620252-f152-4f0c-9dcc-8d50c9669320 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:00.180805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e8620252-f152-4f0c-9dcc-8d50c9669320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:00.234840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c3959-6b04-4f52-b999-2a99db33135e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:00.262511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:00.276376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 143/277] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:00.278246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c70260-0965-4e02-bac5-b658fe32035e None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.278844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c70260-0965-4e02-bac5-b658fe32035e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.279363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c70260-0965-4e02-bac5-b658fe32035e None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.282137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 144/278] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:00.283417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c70260-0965-4e02-bac5-b658fe32035e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:00.294231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3759b-b503-4d51-8f1d-6ef46457afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:00.294820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3759b-b503-4d51-8f1d-6ef46457afac None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.295105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3759b-b503-4d51-8f1d-6ef46457afac None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.295420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3759b-b503-4d51-8f1d-6ef46457afac None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.296147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3759b-b503-4d51-8f1d-6ef46457afac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:00.297619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3759b-b503-4d51-8f1d-6ef46457afac None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:00.387173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 135/279] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:43:00 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:43:00.389770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df12b-3bdf-409c-874d-3d6ce2e67353 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.389770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df12b-3bdf-409c-874d-3d6ce2e67353 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.389770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df12b-3bdf-409c-874d-3d6ce2e67353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.405502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:00.408015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d92df12b-3bdf-409c-874d-3d6ce2e67353 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:00.409323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d92df12b-3bdf-409c-874d-3d6ce2e67353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:00.437048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c70260-0965-4e02-bac5-b658fe32035e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:00.483205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:00.483919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.484202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.484489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.485258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:00.485620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:00.487004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:00.498099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 145/280] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:00.498812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:00.511255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 136/281] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:00.511255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92df12b-3bdf-409c-874d-3d6ce2e67353 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:00.638753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111d20-7a03-4b46-8188-6306ea371e19 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:00.687878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:00.697735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92df12b-3bdf-409c-874d-3d6ce2e67353 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:00.721882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:00.732105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf1cac-8106-4ec4-9209-e527a2a507a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:00.733481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf1cac-8106-4ec4-9209-e527a2a507a0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.733986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf1cac-8106-4ec4-9209-e527a2a507a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.735002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf1cac-8106-4ec4-9209-e527a2a507a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.755440 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf1cac-8106-4ec4-9209-e527a2a507a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:00.755850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2755f-8f7b-4183-b150-6089c42d7b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:00.756721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2755f-8f7b-4183-b150-6089c42d7b18 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.757153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2755f-8f7b-4183-b150-6089c42d7b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.757592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2755f-8f7b-4183-b150-6089c42d7b18 None admin] PATH_INFO: `/v3/users/4162ebb0570646d799b1421fc5c1ada2` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.760083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2755f-8f7b-4183-b150-6089c42d7b18 None admin] RBAC: Authorizing `identity:get_user(user_id=4162ebb0570646d799b1421fc5c1ada2)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:00.761641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2755f-8f7b-4183-b150-6089c42d7b18 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:00.763593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 137/282] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:00 2022] GET /identity/v3/users/4162ebb0570646d799b1421fc5c1ada2 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:00.764580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2755f-8f7b-4183-b150-6089c42d7b18 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0bcf1cac-8106-4ec4-9209-e527a2a507a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:00.765179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:00.766802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:00.775894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 146/283] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:00.777976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf1cac-8106-4ec4-9209-e527a2a507a0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:00.927140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf1cac-8106-4ec4-9209-e527a2a507a0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:00.941519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2755f-8f7b-4183-b150-6089c42d7b18 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:00.965719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f22241-d185-4147-8b2a-473ff2c95e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:00.966625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f22241-d185-4147-8b2a-473ff2c95e4c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:00.966909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f22241-d185-4147-8b2a-473ff2c95e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:00.967204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f22241-d185-4147-8b2a-473ff2c95e4c None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:00.968496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f22241-d185-4147-8b2a-473ff2c95e4c None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:00.969965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f22241-d185-4147-8b2a-473ff2c95e4c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:00.972140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 147/284] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:00 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:00.972658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f22241-d185-4147-8b2a-473ff2c95e4c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:01.017399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803654fa-9a98-4544-85df-b417ec1d0a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:01.018845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803654fa-9a98-4544-85df-b417ec1d0a5a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:01.019433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803654fa-9a98-4544-85df-b417ec1d0a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:01.020137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803654fa-9a98-4544-85df-b417ec1d0a5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:01.041990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803654fa-9a98-4544-85df-b417ec1d0a5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:01.051295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-803654fa-9a98-4544-85df-b417ec1d0a5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:01.059640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:01.067271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 138/285] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:01.068465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803654fa-9a98-4544-85df-b417ec1d0a5a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:01.103444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:01.138385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f22241-d185-4147-8b2a-473ff2c95e4c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:01.193589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803654fa-9a98-4544-85df-b417ec1d0a5a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:01.229175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:01.230335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:01.231184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:01.231709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:01.234093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:01.234641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:01.237234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:01.237401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb87da-c6d3-4238-882d-e815ce59a847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:01.238265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb87da-c6d3-4238-882d-e815ce59a847 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:01.238617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb87da-c6d3-4238-882d-e815ce59a847 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:01.238964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb87da-c6d3-4238-882d-e815ce59a847 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/users/4162ebb0570646d799b1421fc5c1ada2/roles/44ee62ba929b4af4b083efded29d3d5c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:01.241230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb87da-c6d3-4238-882d-e815ce59a847 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc9f58284ad54c079cfbd7d29dbf50ac, user_id=4162ebb0570646d799b1421fc5c1ada2, role_id=44ee62ba929b4af4b083efded29d3d5c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:01.242821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb87da-c6d3-4238-882d-e815ce59a847 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:01.254221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 148/286] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:01.255808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:01.379520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 139/287] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:43:01 2022] PUT /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/users/4162ebb0570646d799b1421fc5c1ada2/roles/44ee62ba929b4af4b083efded29d3d5c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:01.381246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb87da-c6d3-4238-882d-e815ce59a847 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:01.537376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506268cf-4251-4e58-9c64-dd92f5a96694 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:01.618988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:01.623748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:01.655397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb87da-c6d3-4238-882d-e815ce59a847 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:01.664487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:01.665566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:01.666017 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:01.666466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:01.667808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:01.673421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:01.673421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:01.695320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 149/288] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:01.717814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:01.770434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2237d10-f03a-460b-b317-130f902df490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:01.772745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237d10-f03a-460b-b317-130f902df490 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:01.773230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237d10-f03a-460b-b317-130f902df490 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:01.773683 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237d10-f03a-460b-b317-130f902df490 None admin] PATH_INFO: `/v3/users/3d328dc3be0c4f7a99b464e345b6985d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:01.806371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2237d10-f03a-460b-b317-130f902df490 None admin] RBAC: Authorizing `identity:get_user(user_id=3d328dc3be0c4f7a99b464e345b6985d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:01.809060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2237d10-f03a-460b-b317-130f902df490 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:01.813464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 140/289] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:01 2022] GET /identity/v3/users/3d328dc3be0c4f7a99b464e345b6985d => generated 293 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:01.825174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2237d10-f03a-460b-b317-130f902df490 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:01.942839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d318f-f995-4a0b-a0af-f1b7bca334ab None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:02.055631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5052eb6-0e93-4240-a38a-e0736bd8a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:02.057215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5052eb6-0e93-4240-a38a-e0736bd8a274 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:02.057649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5052eb6-0e93-4240-a38a-e0736bd8a274 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:02.058146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5052eb6-0e93-4240-a38a-e0736bd8a274 None admin] PATH_INFO: `/v3/users/23bbb254af924001856937b007559ca9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:02.062867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5052eb6-0e93-4240-a38a-e0736bd8a274 None admin] RBAC: Authorizing `identity:get_user(user_id=23bbb254af924001856937b007559ca9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:02.074139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5052eb6-0e93-4240-a38a-e0736bd8a274 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:02.079187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 150/290] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:01 2022] GET /identity/v3/users/23bbb254af924001856937b007559ca9 => generated 297 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:02.085751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5052eb6-0e93-4240-a38a-e0736bd8a274 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:02.151529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2237d10-f03a-460b-b317-130f902df490 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:02.190643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:02.228734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:02.229416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:02.229696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:02.229972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:02.230895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] RBAC: Adding query filter params (role.id=2e43acd667e44e0bbba45926e652a64f, user.id=3d328dc3be0c4f7a99b464e345b6985d, scope.system=all) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:02.231212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:02.232862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:02.246205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 141/291] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Jul 1 03:43:01 2022] GET /identity/v3/role_assignments?user.id=3d328dc3be0c4f7a99b464e345b6985d&scope.system=all&role.id=2e43acd667e44e0bbba45926e652a64f => generated 226 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:02.284049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5052eb6-0e93-4240-a38a-e0736bd8a274 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:02.342653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae48a00-86d7-4218-95f2-6a5a0e77a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:02.343270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48a00-86d7-4218-95f2-6a5a0e77a667 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:02.343563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48a00-86d7-4218-95f2-6a5a0e77a667 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:02.343839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48a00-86d7-4218-95f2-6a5a0e77a667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:02.453923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae48a00-86d7-4218-95f2-6a5a0e77a667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:02.455453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae48a00-86d7-4218-95f2-6a5a0e77a667 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:02.531591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:02.604307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 151/292] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:02.610134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:02.624735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:02.771495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473682af-0d5a-49fb-be98-221cc838d9fc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:02.820604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b662b-f0fd-4165-942b-f9f606a30681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:02.821507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b662b-f0fd-4165-942b-f9f606a30681 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:02.821797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b662b-f0fd-4165-942b-f9f606a30681 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:02.822081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b662b-f0fd-4165-942b-f9f606a30681 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:02.823410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b662b-f0fd-4165-942b-f9f606a30681 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:02.824902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b662b-f0fd-4165-942b-f9f606a30681 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:02.827080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 142/293] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:02 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:02.833089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae48a00-86d7-4218-95f2-6a5a0e77a667 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:02.967875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae48a00-86d7-4218-95f2-6a5a0e77a667 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:03.013592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52b31b-5681-4b74-8662-15378915d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:03.015186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52b31b-5681-4b74-8662-15378915d86f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:03.016034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52b31b-5681-4b74-8662-15378915d86f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:03.016789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52b31b-5681-4b74-8662-15378915d86f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:03.018726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52b31b-5681-4b74-8662-15378915d86f None admin] RBAC: Adding query filter params (role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac, user.id=23bbb254af924001856937b007559ca9) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:03.019648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52b31b-5681-4b74-8662-15378915d86f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:03.027267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52b31b-5681-4b74-8662-15378915d86f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:03.043806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 152/294] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:02 2022] GET /identity/v3/role_assignments?user.id=23bbb254af924001856937b007559ca9&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=2e43acd667e44e0bbba45926e652a64f => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:03.115557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a056f6b-2b1b-498a-940b-1b14cb2ab0b9 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:03.115843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a056f6b-2b1b-498a-940b-1b14cb2ab0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:03.116934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a056f6b-2b1b-498a-940b-1b14cb2ab0b9 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:03.118648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 143/295] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:03.125586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5e79-edfa-4c08-a341-ecb24a1e56ab None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:03.126101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5e79-edfa-4c08-a341-ecb24a1e56ab None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:03.126494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5e79-edfa-4c08-a341-ecb24a1e56ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:03.167632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e1ff5e79-edfa-4c08-a341-ecb24a1e56ab None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:03.170446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e1ff5e79-edfa-4c08-a341-ecb24a1e56ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:03.195513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:03.327632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 153/296] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:03.363480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a056f6b-2b1b-498a-940b-1b14cb2ab0b9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:03.469503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:03.501498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a056f6b-2b1b-498a-940b-1b14cb2ab0b9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:03.549012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc93fa-d676-420e-8597-92614af38255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:03.549806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc93fa-d676-420e-8597-92614af38255 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:03.550094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc93fa-d676-420e-8597-92614af38255 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:03.550367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc93fa-d676-420e-8597-92614af38255 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:03.563345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc93fa-d676-420e-8597-92614af38255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-17fc93fa-d676-420e-8597-92614af38255 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:03.571408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:03.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:03.574671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 144/297] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:03.585601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff5e79-edfa-4c08-a341-ecb24a1e56ab None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:03.741361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ebb1-740d-41ef-aec2-35535b9e16c3 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:03.741696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ebb1-740d-41ef-aec2-35535b9e16c3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:03.741926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ebb1-740d-41ef-aec2-35535b9e16c3 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:03.743584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 145/298] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:43:03.752184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef993aa-3e7f-4dcd-b05c-9d289211cb43 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:03.752684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef993aa-3e7f-4dcd-b05c-9d289211cb43 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:03.753171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef993aa-3e7f-4dcd-b05c-9d289211cb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:03.773476 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff5e79-edfa-4c08-a341-ecb24a1e56ab None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:03.783257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-aef993aa-3e7f-4dcd-b05c-9d289211cb43 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:03.785643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aef993aa-3e7f-4dcd-b05c-9d289211cb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:03.810738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:03.811457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:03.811741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:03.812016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:03.812782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:03.813091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:03.814612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:03.824575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 154/299] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:03.830221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:03.843558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:03.926261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 146/300] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:03.946579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d42e38-8775-46e2-be03-45e056f6bf75 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:03.965571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef993aa-3e7f-4dcd-b05c-9d289211cb43 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:04.002233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3ed54-12d9-4be2-a0db-ff61fa3bde81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:04.004336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ed54-12d9-4be2-a0db-ff61fa3bde81 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:04.005133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ed54-12d9-4be2-a0db-ff61fa3bde81 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:04.005869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ed54-12d9-4be2-a0db-ff61fa3bde81 None admin] PATH_INFO: `/v3/users/2df99636cf414cc2b10b9ea16b25c25d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:04.033702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3ed54-12d9-4be2-a0db-ff61fa3bde81 None admin] RBAC: Authorizing `identity:get_user(user_id=2df99636cf414cc2b10b9ea16b25c25d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:04.036932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3ed54-12d9-4be2-a0db-ff61fa3bde81 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:04.040964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 155/301] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:03 2022] GET /identity/v3/users/2df99636cf414cc2b10b9ea16b25c25d => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:04.046202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3ed54-12d9-4be2-a0db-ff61fa3bde81 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:04.093487 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:04.141026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef993aa-3e7f-4dcd-b05c-9d289211cb43 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:04.188912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:04.202820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e4c2-877d-4525-ac87-d0637ee28c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:04.204912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e4c2-877d-4525-ac87-d0637ee28c50 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:04.205850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e4c2-877d-4525-ac87-d0637ee28c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:04.206373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e4c2-877d-4525-ac87-d0637ee28c50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:04.209263 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2e4c2-877d-4525-ac87-d0637ee28c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:04.213304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2e4c2-877d-4525-ac87-d0637ee28c50 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:04.216922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 147/302] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:04.229649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e4c2-877d-4525-ac87-d0637ee28c50 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:04.244563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3ed54-12d9-4be2-a0db-ff61fa3bde81 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:04.321209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864252f-62de-491f-ab3c-f7b233869782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:04.322362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864252f-62de-491f-ab3c-f7b233869782 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:04.322851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864252f-62de-491f-ab3c-f7b233869782 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:04.323363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864252f-62de-491f-ab3c-f7b233869782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:04.325191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864252f-62de-491f-ab3c-f7b233869782 None admin] RBAC: Adding query filter params (role.id=44ee62ba929b4af4b083efded29d3d5c, user.id=2df99636cf414cc2b10b9ea16b25c25d, scope.system=all) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:04.326313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864252f-62de-491f-ab3c-f7b233869782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:04.329360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864252f-62de-491f-ab3c-f7b233869782 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:04.341616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 156/303] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Jul 1 03:43:04 2022] GET /identity/v3/role_assignments?user.id=2df99636cf414cc2b10b9ea16b25c25d&scope.system=all&role.id=44ee62ba929b4af4b083efded29d3d5c => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:04.441750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e4c2-877d-4525-ac87-d0637ee28c50 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:04.511887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efe1df-3039-46f3-88c0-829fb9d82c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:04.512595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efe1df-3039-46f3-88c0-829fb9d82c59 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:04.512878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efe1df-3039-46f3-88c0-829fb9d82c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:04.513148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efe1df-3039-46f3-88c0-829fb9d82c59 None admin] PATH_INFO: `/v3/groups` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:04.513890 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efe1df-3039-46f3-88c0-829fb9d82c59 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:04.515499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efe1df-3039-46f3-88c0-829fb9d82c59 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:04.518901 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:04.595945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 148/304] 10.222.0.20 () {60 vars in 1134 bytes} [Fri Jul 1 03:43:04 2022] POST /identity/v3/groups => generated 229 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:43:04.644698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b625d-2d93-48f1-96c0-d3d87c3aca66 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:04.645155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b625d-2d93-48f1-96c0-d3d87c3aca66 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:04.645617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b625d-2d93-48f1-96c0-d3d87c3aca66 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:04.648498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 157/305] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:04.657426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae1f9-cc13-4f7c-ba3b-0365dd6276c4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:04.657886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae1f9-cc13-4f7c-ba3b-0365dd6276c4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:04.658344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae1f9-cc13-4f7c-ba3b-0365dd6276c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:04.688136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-660ae1f9-cc13-4f7c-ba3b-0365dd6276c4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:04.690177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-660ae1f9-cc13-4f7c-ba3b-0365dd6276c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:04.813412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 149/306] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:04.849241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b625d-2d93-48f1-96c0-d3d87c3aca66 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:04.940912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4928-e9e2-418d-a1dc-ce33deb31690 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:04.941239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4928-e9e2-418d-a1dc-ce33deb31690 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:04.941481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf4928-e9e2-418d-a1dc-ce33deb31690 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:04.943026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 150/307] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:43:04.949650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888dc6-191f-4e24-bbc2-daa952bbda55 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:04.950155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888dc6-191f-4e24-bbc2-daa952bbda55 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:04.950636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888dc6-191f-4e24-bbc2-daa952bbda55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:04.983174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-39888dc6-191f-4e24-bbc2-daa952bbda55 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:04.986523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-39888dc6-191f-4e24-bbc2-daa952bbda55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:05.000554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b625d-2d93-48f1-96c0-d3d87c3aca66 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:05.039732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:05.047471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e069af-1459-41b6-a7eb-622d6016fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:05.048918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e069af-1459-41b6-a7eb-622d6016fc83 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:05.055387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e069af-1459-41b6-a7eb-622d6016fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:05.055387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e069af-1459-41b6-a7eb-622d6016fc83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:05.066426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e069af-1459-41b6-a7eb-622d6016fc83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-83e069af-1459-41b6-a7eb-622d6016fc83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:05.077103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:05.078085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:05.078860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:05.081958 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 158/308] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:05.091256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e069af-1459-41b6-a7eb-622d6016fc83 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:05.120226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 151/309] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:05.150027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888dc6-191f-4e24-bbc2-daa952bbda55 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:05.215049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:05.269504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888dc6-191f-4e24-bbc2-daa952bbda55 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:05.293888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e069af-1459-41b6-a7eb-622d6016fc83 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:05.313191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d6282-8d73-4f9a-b260-2bff46ee4d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:05.314144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6282-8d73-4f9a-b260-2bff46ee4d86 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:05.314531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6282-8d73-4f9a-b260-2bff46ee4d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:05.314897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6282-8d73-4f9a-b260-2bff46ee4d86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:05.327320 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d6282-8d73-4f9a-b260-2bff46ee4d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3f0d6282-8d73-4f9a-b260-2bff46ee4d86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:05.336137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:05.337248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:05.338163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:05.339284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 152/310] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:43:05.345657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d6282-8d73-4f9a-b260-2bff46ee4d86 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:05.358292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:05.359415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:05.359899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:05.360413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:05.362184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:05.362697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:05.364912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:05.376708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 159/311] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:05.382898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:05.484312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d6282-8d73-4f9a-b260-2bff46ee4d86 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:05.501001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:05.515772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:05.559799 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00787c-3554-49ea-91d4-ed7e8185e59b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:05.566137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:05.566880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:05.567173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:05.567499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:05.568417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:05.568729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:05.570686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:05.582065 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 153/312] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:05.588437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:05.638272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c6b37-6ec0-4aed-b5a7-d6b9762cf8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:05.640857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c6b37-6ec0-4aed-b5a7-d6b9762cf8ee None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:05.641591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c6b37-6ec0-4aed-b5a7-d6b9762cf8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:05.642777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c6b37-6ec0-4aed-b5a7-d6b9762cf8ee None admin] PATH_INFO: `/v3/users/4162ebb0570646d799b1421fc5c1ada2` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:05.646044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c6b37-6ec0-4aed-b5a7-d6b9762cf8ee None admin] RBAC: Authorizing `identity:get_user(user_id=4162ebb0570646d799b1421fc5c1ada2)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:05.650424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c6b37-6ec0-4aed-b5a7-d6b9762cf8ee None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:05.655289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 160/313] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:05 2022] GET /identity/v3/users/4162ebb0570646d799b1421fc5c1ada2 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:05.661641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c6b37-6ec0-4aed-b5a7-d6b9762cf8ee None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:05.732526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193dfb-18a8-4464-9613-a77f047e2c2d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:05.780544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f79d23-4431-4dcd-b108-e91448293089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:05.781516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f79d23-4431-4dcd-b108-e91448293089 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:05.782006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f79d23-4431-4dcd-b108-e91448293089 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:05.782485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f79d23-4431-4dcd-b108-e91448293089 None admin] PATH_INFO: `/v3/users/3d328dc3be0c4f7a99b464e345b6985d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:05.784712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f79d23-4431-4dcd-b108-e91448293089 None admin] RBAC: Authorizing `identity:get_user(user_id=3d328dc3be0c4f7a99b464e345b6985d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:05.787693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f79d23-4431-4dcd-b108-e91448293089 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:05.797686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 154/314] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:05 2022] GET /identity/v3/users/3d328dc3be0c4f7a99b464e345b6985d => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:05.801782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f79d23-4431-4dcd-b108-e91448293089 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:05.815724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c6b37-6ec0-4aed-b5a7-d6b9762cf8ee None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:05.841713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:05.859095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a727b19-e632-4739-a9ff-bc7a30f07ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:05.859984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a727b19-e632-4739-a9ff-bc7a30f07ccb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:05.860364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a727b19-e632-4739-a9ff-bc7a30f07ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:05.860763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a727b19-e632-4739-a9ff-bc7a30f07ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:05.871625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:05.959047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f79d23-4431-4dcd-b108-e91448293089 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:05.988847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a727b19-e632-4739-a9ff-bc7a30f07ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:05.993566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a727b19-e632-4739-a9ff-bc7a30f07ccb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:06.026545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a790c-f5f3-469d-ae61-e67d16ee7251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:06.028696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a790c-f5f3-469d-ae61-e67d16ee7251 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:06.029124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a790c-f5f3-469d-ae61-e67d16ee7251 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:06.029532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a790c-f5f3-469d-ae61-e67d16ee7251 None admin] PATH_INFO: `/v3/system/users/3d328dc3be0c4f7a99b464e345b6985d/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:06.031675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a790c-f5f3-469d-ae61-e67d16ee7251 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3d328dc3be0c4f7a99b464e345b6985d, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:06.033207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a790c-f5f3-469d-ae61-e67d16ee7251 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:06.141157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 155/315] 10.222.0.20 () {58 vars in 1334 bytes} [Fri Jul 1 03:43:05 2022] PUT /identity/v3/system/users/3d328dc3be0c4f7a99b464e345b6985d/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 03:43:06.201030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 161/316] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 03:43:06.203640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a790c-f5f3-469d-ae61-e67d16ee7251 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:06.407961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a790c-f5f3-469d-ae61-e67d16ee7251 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:06.428940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:06.483410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb07831-dc33-4791-b59c-32a776cd1b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:06.485146 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb07831-dc33-4791-b59c-32a776cd1b85 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:06.485909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb07831-dc33-4791-b59c-32a776cd1b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:06.486147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb07831-dc33-4791-b59c-32a776cd1b85 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:06.488442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb07831-dc33-4791-b59c-32a776cd1b85 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:06.491788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb07831-dc33-4791-b59c-32a776cd1b85 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:06.495874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 156/317] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:06 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:06.504162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a727b19-e632-4739-a9ff-bc7a30f07ccb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:06.530317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:06.607689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e59fcd-641d-4efc-ab2f-90c8a6ad44ce None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:06.608197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e59fcd-641d-4efc-ab2f-90c8a6ad44ce None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:06.608694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e59fcd-641d-4efc-ab2f-90c8a6ad44ce None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:06.611460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 157/318] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:06.619902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cc80-a0e8-4bc5-900c-06fed1d6af40 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:06.620186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cc80-a0e8-4bc5-900c-06fed1d6af40 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:06.620463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cc80-a0e8-4bc5-900c-06fed1d6af40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:06.625002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a727b19-e632-4739-a9ff-bc7a30f07ccb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:06.649434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dfc9cc80-a0e8-4bc5-900c-06fed1d6af40 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:06.651829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dfc9cc80-a0e8-4bc5-900c-06fed1d6af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:06.666787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:06.667747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:06.668179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:06.668589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:06.669592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] RBAC: Adding query filter params (role.id=44ee62ba929b4af4b083efded29d3d5c, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac, user.id=4162ebb0570646d799b1421fc5c1ada2) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:06.670054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:06.671714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:06.679200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 162/319] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:06 2022] GET /identity/v3/role_assignments?user.id=4162ebb0570646d799b1421fc5c1ada2&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=44ee62ba929b4af4b083efded29d3d5c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:06.777634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 158/320] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:06.812074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:06.926955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:06.935909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae551-d06f-4eea-a6c7-c52a6ab62e6a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:06.936192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae551-d06f-4eea-a6c7-c52a6ab62e6a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:06.936610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae551-d06f-4eea-a6c7-c52a6ab62e6a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:06.938253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 159/321] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:06.943880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee55a0-5cff-42e8-91d6-cb73766dfe68 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:06.944793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee55a0-5cff-42e8-91d6-cb73766dfe68 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:06.945028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee55a0-5cff-42e8-91d6-cb73766dfe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:06.946742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc5f4-cc65-4a7b-a2f6-74bf7e68fde0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:06.976217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-65ee55a0-5cff-42e8-91d6-cb73766dfe68 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:06.978557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-65ee55a0-5cff-42e8-91d6-cb73766dfe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:07.002084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96128f9-0655-4f85-9a8e-35591938aef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:07.002926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96128f9-0655-4f85-9a8e-35591938aef8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:07.003208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96128f9-0655-4f85-9a8e-35591938aef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:07.003517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96128f9-0655-4f85-9a8e-35591938aef8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:07.024957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96128f9-0655-4f85-9a8e-35591938aef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c96128f9-0655-4f85-9a8e-35591938aef8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:07.039977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:07.040943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:07.041717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:07.042685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 163/322] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:07.051157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96128f9-0655-4f85-9a8e-35591938aef8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:07.095262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 160/323] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:07.133268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee55a0-5cff-42e8-91d6-cb73766dfe68 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:07.185008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:07.205029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96128f9-0655-4f85-9a8e-35591938aef8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:07.267853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:07.268708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:07.269206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:07.269609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:07.270705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:07.271877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:07.274805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:07.288287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee55a0-5cff-42e8-91d6-cb73766dfe68 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:07.292370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 164/324] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:07.298332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:07.311817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:07.354988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025cc7-93e4-4cd7-ad38-cb24acd3fde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:07.356313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025cc7-93e4-4cd7-ad38-cb24acd3fde2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:07.356790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025cc7-93e4-4cd7-ad38-cb24acd3fde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:07.357263 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025cc7-93e4-4cd7-ad38-cb24acd3fde2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:07.359423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90025cc7-93e4-4cd7-ad38-cb24acd3fde2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:07.362019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90025cc7-93e4-4cd7-ad38-cb24acd3fde2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:07.365260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 161/325] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:07.374517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025cc7-93e4-4cd7-ad38-cb24acd3fde2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:07.446424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4475ae-7b98-46c4-840e-e64d662e46c7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:07.506066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea01518-82b6-4f12-b863-cf734e9824ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:07.507649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea01518-82b6-4f12-b863-cf734e9824ef None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:07.508128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea01518-82b6-4f12-b863-cf734e9824ef None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:07.508622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea01518-82b6-4f12-b863-cf734e9824ef None admin] PATH_INFO: `/v3/users/2df99636cf414cc2b10b9ea16b25c25d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:07.510816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea01518-82b6-4f12-b863-cf734e9824ef None admin] RBAC: Authorizing `identity:get_user(user_id=2df99636cf414cc2b10b9ea16b25c25d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:07.514492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea01518-82b6-4f12-b863-cf734e9824ef None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:07.519604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 165/326] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:07 2022] GET /identity/v3/users/2df99636cf414cc2b10b9ea16b25c25d => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:07.526859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea01518-82b6-4f12-b863-cf734e9824ef None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:07.559406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025cc7-93e4-4cd7-ad38-cb24acd3fde2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:07.565326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:07.623810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cec86a-b0b9-45db-a3aa-9afb2e812743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:07.624487 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec86a-b0b9-45db-a3aa-9afb2e812743 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:07.624823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec86a-b0b9-45db-a3aa-9afb2e812743 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:07.625107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec86a-b0b9-45db-a3aa-9afb2e812743 None admin] PATH_INFO: `/v3/groups` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:07.625828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cec86a-b0b9-45db-a3aa-9afb2e812743 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:07.627382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cec86a-b0b9-45db-a3aa-9afb2e812743 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:07.639593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:07.682710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea01518-82b6-4f12-b863-cf734e9824ef None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:07.744428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 162/327] 10.222.0.20 () {60 vars in 1134 bytes} [Fri Jul 1 03:43:07 2022] POST /identity/v3/groups => generated 226 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:43:07.748617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbbe74-7e34-4125-8c5f-a717fd5f12bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:07.750038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbbe74-7e34-4125-8c5f-a717fd5f12bc None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:07.750509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbbe74-7e34-4125-8c5f-a717fd5f12bc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:07.750979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbbe74-7e34-4125-8c5f-a717fd5f12bc None admin] PATH_INFO: `/v3/system/users/2df99636cf414cc2b10b9ea16b25c25d/roles/44ee62ba929b4af4b083efded29d3d5c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:07.753728 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbbe74-7e34-4125-8c5f-a717fd5f12bc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2df99636cf414cc2b10b9ea16b25c25d, role_id=44ee62ba929b4af4b083efded29d3d5c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:07.756298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbbe74-7e34-4125-8c5f-a717fd5f12bc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:07.830849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 166/328] 10.222.0.20 () {58 vars in 1334 bytes} [Fri Jul 1 03:43:07 2022] PUT /identity/v3/system/users/2df99636cf414cc2b10b9ea16b25c25d/roles/44ee62ba929b4af4b083efded29d3d5c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 03:43:08.695313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a596e-7b58-4250-a158-12f2d8705ddb None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:08.696421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a596e-7b58-4250-a158-12f2d8705ddb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:08.696920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a596e-7b58-4250-a158-12f2d8705ddb None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:08.698692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 163/329] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:08.706854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c721-0bcf-438c-a003-5307c68fe9cb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:08.707838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c721-0bcf-438c-a003-5307c68fe9cb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:08.708311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c721-0bcf-438c-a003-5307c68fe9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:08.733909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-62a1c721-0bcf-438c-a003-5307c68fe9cb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:08.735460 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-62a1c721-0bcf-438c-a003-5307c68fe9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:08.836693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 167/330] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:08.886840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a596e-7b58-4250-a158-12f2d8705ddb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:09.059680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a596e-7b58-4250-a158-12f2d8705ddb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:09.106965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e77fe7-d733-4724-a94d-15374a800d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:09.107845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e77fe7-d733-4724-a94d-15374a800d90 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:09.108129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e77fe7-d733-4724-a94d-15374a800d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:09.108404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e77fe7-d733-4724-a94d-15374a800d90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:09.125429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e77fe7-d733-4724-a94d-15374a800d90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-12e77fe7-d733-4724-a94d-15374a800d90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:09.133564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:09.134461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:09.135718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 164/331] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:09.144022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1c721-0bcf-438c-a003-5307c68fe9cb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:09.284867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:09.355497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1c721-0bcf-438c-a003-5307c68fe9cb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:09.418415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:09.419095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:09.419401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:09.419677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:09.420453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:09.420815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:09.422258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:09.435353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 168/332] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:09.437844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e77fe7-d733-4724-a94d-15374a800d90 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:09.479591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:09.583042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e77fe7-d733-4724-a94d-15374a800d90 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:09.632563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39442fd6-7930-4378-b5d8-37d05f9b6404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:09.633812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39442fd6-7930-4378-b5d8-37d05f9b6404 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:09.634298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39442fd6-7930-4378-b5d8-37d05f9b6404 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:09.634770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39442fd6-7930-4378-b5d8-37d05f9b6404 None admin] PATH_INFO: `/v3/users/3d328dc3be0c4f7a99b464e345b6985d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:09.636287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39442fd6-7930-4378-b5d8-37d05f9b6404 None admin] RBAC: Authorizing `identity:get_user(user_id=3d328dc3be0c4f7a99b464e345b6985d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:09.637986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39442fd6-7930-4378-b5d8-37d05f9b6404 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:09.640201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 165/333] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:09 2022] GET /identity/v3/users/3d328dc3be0c4f7a99b464e345b6985d => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:09.645908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:09.808319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172953e-afa7-46bf-b554-bd6db505f0f9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:09.851699 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:09.862639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86ffbf-d9c2-4965-b580-985bcb26528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:09.863357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86ffbf-d9c2-4965-b580-985bcb26528b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:09.863645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86ffbf-d9c2-4965-b580-985bcb26528b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:09.863923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86ffbf-d9c2-4965-b580-985bcb26528b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:09.864805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86ffbf-d9c2-4965-b580-985bcb26528b None admin] RBAC: Adding query filter params (role.id=2e43acd667e44e0bbba45926e652a64f, user.id=3d328dc3be0c4f7a99b464e345b6985d, scope.system=all) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:09.865119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86ffbf-d9c2-4965-b580-985bcb26528b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:09.866739 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86ffbf-d9c2-4965-b580-985bcb26528b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:09.884464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 169/334] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Jul 1 03:43:09 2022] GET /identity/v3/role_assignments?user.id=3d328dc3be0c4f7a99b464e345b6985d&scope.system=all&role.id=2e43acd667e44e0bbba45926e652a64f => generated 510 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:10.987594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca723e8-2251-4702-bba7-e9e95430faa9 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:10.988175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca723e8-2251-4702-bba7-e9e95430faa9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:10.988432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca723e8-2251-4702-bba7-e9e95430faa9 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:10.994298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 166/335] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:11.001483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a97fb-7f05-4f14-8d7a-f6f258db21fd None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:11.002317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a97fb-7f05-4f14-8d7a-f6f258db21fd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:11.002759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a97fb-7f05-4f14-8d7a-f6f258db21fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:11.028309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b13a97fb-7f05-4f14-8d7a-f6f258db21fd None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:11.029541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b13a97fb-7f05-4f14-8d7a-f6f258db21fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:11.122772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 170/336] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:11.199842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca723e8-2251-4702-bba7-e9e95430faa9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:11.313566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:11.326069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca723e8-2251-4702-bba7-e9e95430faa9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:11.372525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764c612-25c0-4531-bb0d-80dece39136a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:11.373347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764c612-25c0-4531-bb0d-80dece39136a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:11.373636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764c612-25c0-4531-bb0d-80dece39136a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:11.373918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764c612-25c0-4531-bb0d-80dece39136a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:11.395434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764c612-25c0-4531-bb0d-80dece39136a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b764c612-25c0-4531-bb0d-80dece39136a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:11.403986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:11.404934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:11.405723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 167/337] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:11.414537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a97fb-7f05-4f14-8d7a-f6f258db21fd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:11.539125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a97fb-7f05-4f14-8d7a-f6f258db21fd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:11.545732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593d4a0-fa2b-4978-a9f0-ad8648a4db5a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:11.546039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593d4a0-fa2b-4978-a9f0-ad8648a4db5a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:11.546285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593d4a0-fa2b-4978-a9f0-ad8648a4db5a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:11.547943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 168/338] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:11.554381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eb430-064b-4fdd-9497-1ba040047f6f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:11.556254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eb430-064b-4fdd-9497-1ba040047f6f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:11.556753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eb430-064b-4fdd-9497-1ba040047f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:11.581365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:11.585874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fd7eb430-064b-4fdd-9497-1ba040047f6f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:11.588877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fd7eb430-064b-4fdd-9497-1ba040047f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:11.623771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:11.626509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:11.627028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:11.627547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:11.628551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:11.629066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:11.630650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:11.647719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 171/339] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:11.656830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:11.753884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 169/340] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:11.763884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128ec4b-71a1-4169-b40a-9cd1e484f295 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:11.764836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128ec4b-71a1-4169-b40a-9cd1e484f295 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:11.765661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128ec4b-71a1-4169-b40a-9cd1e484f295 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:11.769050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 170/341] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:11.775706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c4d22-db6c-4392-89d8-90382aa536d7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:11.775976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c4d22-db6c-4392-89d8-90382aa536d7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:11.776407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c4d22-db6c-4392-89d8-90382aa536d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:11.797034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-226c4d22-db6c-4392-89d8-90382aa536d7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:11.798744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-226c4d22-db6c-4392-89d8-90382aa536d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:11.891785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609c095-d9eb-4042-ba46-9619ae78752b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:11.919603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 171/342] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:11.920371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c4d22-db6c-4392-89d8-90382aa536d7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:11.947795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:11.966400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48534e4-2bd5-415a-99fd-beccf2409b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:11.967310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48534e4-2bd5-415a-99fd-beccf2409b15 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:11.967592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48534e4-2bd5-415a-99fd-beccf2409b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:11.967865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48534e4-2bd5-415a-99fd-beccf2409b15 None admin] PATH_INFO: `/v3/users/2df99636cf414cc2b10b9ea16b25c25d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:11.970026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48534e4-2bd5-415a-99fd-beccf2409b15 None admin] RBAC: Authorizing `identity:get_user(user_id=2df99636cf414cc2b10b9ea16b25c25d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:11.971844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48534e4-2bd5-415a-99fd-beccf2409b15 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:11.981320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 172/343] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:43:11 2022] GET /identity/v3/users/2df99636cf414cc2b10b9ea16b25c25d => generated 293 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:11.983258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48534e4-2bd5-415a-99fd-beccf2409b15 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:12.094999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:12.112540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c4d22-db6c-4392-89d8-90382aa536d7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:12.159714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2366-2f2f-4487-9096-494019ddcb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:12.160810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2366-2f2f-4487-9096-494019ddcb0b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:12.161249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2366-2f2f-4487-9096-494019ddcb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:12.161662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2366-2f2f-4487-9096-494019ddcb0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:12.178995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2366-2f2f-4487-9096-494019ddcb0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-c2fb2366-2f2f-4487-9096-494019ddcb0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:12.187650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:12.188656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:12.189463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:12.191881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 172/344] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:12.192994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2366-2f2f-4487-9096-494019ddcb0b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:12.205933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48534e4-2bd5-415a-99fd-beccf2409b15 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:12.289727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc9149-f2b4-45e0-aeb7-384cbb27ef41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:12.291604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9149-f2b4-45e0-aeb7-384cbb27ef41 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:12.292574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9149-f2b4-45e0-aeb7-384cbb27ef41 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:12.293353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9149-f2b4-45e0-aeb7-384cbb27ef41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:12.302406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:12.326708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc9149-f2b4-45e0-aeb7-384cbb27ef41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-00bc9149-f2b4-45e0-aeb7-384cbb27ef41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:12.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:12.349020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:12.351180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:12.358731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 173/345] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:12.361607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb6df-7f11-49cb-a0da-5758a391eb59 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:12.362532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb6df-7f11-49cb-a0da-5758a391eb59 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:12.363362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb6df-7f11-49cb-a0da-5758a391eb59 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:12.366282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 174/346] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:12.368060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb6df-7f11-49cb-a0da-5758a391eb59 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:12.371561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2366-2f2f-4487-9096-494019ddcb0b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:12.457252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:12.460234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:12.461634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:12.462948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:12.467384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] RBAC: Adding query filter params (role.id=44ee62ba929b4af4b083efded29d3d5c, user.id=2df99636cf414cc2b10b9ea16b25c25d, scope.system=all) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:12.467384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:12.468554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:12.501002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 173/347] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Jul 1 03:43:12 2022] GET /identity/v3/role_assignments?user.id=2df99636cf414cc2b10b9ea16b25c25d&scope.system=all&role.id=44ee62ba929b4af4b083efded29d3d5c => generated 510 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:12.502983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:12.573499 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb6df-7f11-49cb-a0da-5758a391eb59 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:12.598620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:12.661009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b8d8f-0016-4e1f-9328-15b601310bce None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:12.710305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:12.717642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53e808-a85d-4c7b-9563-90eb5af88cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:12.718450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e808-a85d-4c7b-9563-90eb5af88cfb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:12.724692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e808-a85d-4c7b-9563-90eb5af88cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:12.724692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e808-a85d-4c7b-9563-90eb5af88cfb None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:12.724692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53e808-a85d-4c7b-9563-90eb5af88cfb None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:12.724692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53e808-a85d-4c7b-9563-90eb5af88cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:12.725206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:12.725383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53e808-a85d-4c7b-9563-90eb5af88cfb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:12.725533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:12.725533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:12.725533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:12.725533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:12.725533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:12.727743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:12.737044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 174/348] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:12.739090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e1bd7-22d1-4ac0-b837-9600c9a70f4f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:12.743675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e1bd7-22d1-4ac0-b837-9600c9a70f4f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:12.744213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e1bd7-22d1-4ac0-b837-9600c9a70f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:12.750153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 175/349] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:12.751606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:12.799664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-456e1bd7-22d1-4ac0-b837-9600c9a70f4f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:12.801651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-456e1bd7-22d1-4ac0-b837-9600c9a70f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:12.982824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 175/350] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:12.988461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e1bd7-22d1-4ac0-b837-9600c9a70f4f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:13.054284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc10a5c-9c7f-4ff6-b875-7c50ea49ae4d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:13.144387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeba57-8dfe-4165-965d-6f3c94532555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:13.145566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeba57-8dfe-4165-965d-6f3c94532555 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:13.145975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeba57-8dfe-4165-965d-6f3c94532555 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:13.146464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeba57-8dfe-4165-965d-6f3c94532555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:13.243561 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:13.248580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e1bd7-22d1-4ac0-b837-9600c9a70f4f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:13.304280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:13.315462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeba57-8dfe-4165-965d-6f3c94532555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:13.320454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeba57-8dfe-4165-965d-6f3c94532555 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:13.334787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd92e8-f5ea-4619-8833-258f14bcd3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:13.336184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd92e8-f5ea-4619-8833-258f14bcd3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:13.336816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd92e8-f5ea-4619-8833-258f14bcd3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:13.337429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd92e8-f5ea-4619-8833-258f14bcd3b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:13.478963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd92e8-f5ea-4619-8833-258f14bcd3b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:13.480469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd92e8-f5ea-4619-8833-258f14bcd3b4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:13.559470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 176/351] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 809 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:13.561648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeba57-8dfe-4165-965d-6f3c94532555 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:13.645412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 176/352] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:13.647477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550422-702f-479e-a1dd-20bc30707a15 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:13.647943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550422-702f-479e-a1dd-20bc30707a15 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:13.648427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550422-702f-479e-a1dd-20bc30707a15 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:13.651025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 177/353] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:13.654941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03550422-702f-479e-a1dd-20bc30707a15 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:13.676474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:13.702941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:13.729733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeba57-8dfe-4165-965d-6f3c94532555 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:13.786430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067b5af-1f01-43d0-bd96-2ee1b1ca2cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:13.787394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067b5af-1f01-43d0-bd96-2ee1b1ca2cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:13.788322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067b5af-1f01-43d0-bd96-2ee1b1ca2cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:13.788601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067b5af-1f01-43d0-bd96-2ee1b1ca2cf8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:13.800433 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067b5af-1f01-43d0-bd96-2ee1b1ca2cf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1067b5af-1f01-43d0-bd96-2ee1b1ca2cf8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:13.808935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:13.809941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:13.810734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 177/354] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:43:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:13.812953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067b5af-1f01-43d0-bd96-2ee1b1ca2cf8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:13.855892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03550422-702f-479e-a1dd-20bc30707a15 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:13.909718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a9acc-8ee3-4a41-bf0b-8b06d785de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:13.911260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9acc-8ee3-4a41-bf0b-8b06d785de8f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:13.912350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9acc-8ee3-4a41-bf0b-8b06d785de8f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:13.912350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9acc-8ee3-4a41-bf0b-8b06d785de8f None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:13.915360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a9acc-8ee3-4a41-bf0b-8b06d785de8f None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:13.918552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a9acc-8ee3-4a41-bf0b-8b06d785de8f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:13.922915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 178/355] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:13 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:13.926063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699551-c7b4-410c-a974-678ff88041bc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:13.926063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699551-c7b4-410c-a974-678ff88041bc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:13.926063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699551-c7b4-410c-a974-678ff88041bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:13.954929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2e699551-c7b4-410c-a974-678ff88041bc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:13.957349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2e699551-c7b4-410c-a974-678ff88041bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:13.989352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067b5af-1f01-43d0-bd96-2ee1b1ca2cf8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:14.049230 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:14.058889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f422297-8546-49bb-bc7c-b3e5b1fbf4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:14.065418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422297-8546-49bb-bc7c-b3e5b1fbf4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:14.065418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422297-8546-49bb-bc7c-b3e5b1fbf4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:14.065418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422297-8546-49bb-bc7c-b3e5b1fbf4f8 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:14.065418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f422297-8546-49bb-bc7c-b3e5b1fbf4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:14.066538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f422297-8546-49bb-bc7c-b3e5b1fbf4f8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:14.070483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 178/356] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:13 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:14.072535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103e9b1-3a9f-4a1b-9753-51667fe84418 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:14.075055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103e9b1-3a9f-4a1b-9753-51667fe84418 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:14.075055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103e9b1-3a9f-4a1b-9753-51667fe84418 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:14.077176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 179/357] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:14.077424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103e9b1-3a9f-4a1b-9753-51667fe84418 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:14.087359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 179/358] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:14.088605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699551-c7b4-410c-a974-678ff88041bc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:14.125763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:14.265372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103e9b1-3a9f-4a1b-9753-51667fe84418 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:14.286691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699551-c7b4-410c-a974-678ff88041bc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:14.322104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:14.322709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:14.322985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:14.323332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:14.324205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:14.324513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:14.326329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:14.341416 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 180/359] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:14.345032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:14.353765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2f89b-7430-4df7-b864-55e919f57dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:14.355141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2f89b-7430-4df7-b864-55e919f57dab None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:14.355666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2f89b-7430-4df7-b864-55e919f57dab None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:14.356109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2f89b-7430-4df7-b864-55e919f57dab None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:14.367184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2f89b-7430-4df7-b864-55e919f57dab None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:14.369769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2f89b-7430-4df7-b864-55e919f57dab None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:14.372960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 180/360] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:14 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:14.374116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa1809-a0df-4d80-b2e5-e7021ab91bbf None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:14.374598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa1809-a0df-4d80-b2e5-e7021ab91bbf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:14.374877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa1809-a0df-4d80-b2e5-e7021ab91bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:14.407958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c4fa1809-a0df-4d80-b2e5-e7021ab91bbf None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:14.410096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c4fa1809-a0df-4d80-b2e5-e7021ab91bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:14.445395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:14.517317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 181/361] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:14.518564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa1809-a0df-4d80-b2e5-e7021ab91bbf None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:14.524825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefd9d8-fcc2-4581-9957-577815a16e2e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:14.539048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:14.591883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4232bf-b3f8-4918-928b-3b85d06e6e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:14.592833 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4232bf-b3f8-4918-928b-3b85d06e6e64 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:14.593255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4232bf-b3f8-4918-928b-3b85d06e6e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:14.593618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4232bf-b3f8-4918-928b-3b85d06e6e64 None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:14.596355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4232bf-b3f8-4918-928b-3b85d06e6e64 None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:14.597525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4232bf-b3f8-4918-928b-3b85d06e6e64 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:14.599784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 181/362] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:14 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:14.603394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4232bf-b3f8-4918-928b-3b85d06e6e64 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:14.689032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa1809-a0df-4d80-b2e5-e7021ab91bbf None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:14.757284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39c0e-6c5b-4387-ba69-2d581a90904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:14.759069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39c0e-6c5b-4387-ba69-2d581a90904c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:14.759832 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39c0e-6c5b-4387-ba69-2d581a90904c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:14.760118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39c0e-6c5b-4387-ba69-2d581a90904c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:14.781660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d39c0e-6c5b-4387-ba69-2d581a90904c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:14.790663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4232bf-b3f8-4918-928b-3b85d06e6e64 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-21d39c0e-6c5b-4387-ba69-2d581a90904c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:14.796999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:14.798737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 182/363] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:43:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:14.800311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39c0e-6c5b-4387-ba69-2d581a90904c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:14.838273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:14.856602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fa736-53ba-4a83-9767-0856a39092dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:14.858032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fa736-53ba-4a83-9767-0856a39092dc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:14.858583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fa736-53ba-4a83-9767-0856a39092dc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:14.858947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fa736-53ba-4a83-9767-0856a39092dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:14.939479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:14.984015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fa736-53ba-4a83-9767-0856a39092dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:14.986493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fa736-53ba-4a83-9767-0856a39092dc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:14.988018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39c0e-6c5b-4387-ba69-2d581a90904c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:15.057334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:15.058382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:15.058818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:15.059298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:15.060781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] RBAC: Adding query filter params (group.id=d3c7e9ff657443c59a02bf3c94b3416f, role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=e432781ae1bf4d319a43f8749c32be0f) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:15.061290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:15.064599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:15.075841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 183/364] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:14 2022] GET /identity/v3/role_assignments?group.id=d3c7e9ff657443c59a02bf3c94b3416f&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=2e43acd667e44e0bbba45926e652a64f => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:15.077745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:15.192832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 182/365] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:15.194664 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fa736-53ba-4a83-9767-0856a39092dc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:15.250306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:15.291257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ffc9c-0329-4df0-944d-a96f8efb93b4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:15.350781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c895e2-bdc7-4508-a0be-42692c404a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:15.351635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c895e2-bdc7-4508-a0be-42692c404a5d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:15.351915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c895e2-bdc7-4508-a0be-42692c404a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:15.352193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c895e2-bdc7-4508-a0be-42692c404a5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:15.354125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fa736-53ba-4a83-9767-0856a39092dc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:15.371173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:15.376095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c895e2-bdc7-4508-a0be-42692c404a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-36c895e2-bdc7-4508-a0be-42692c404a5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:15.384130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:15.390063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 184/366] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:43:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:15.391504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c895e2-bdc7-4508-a0be-42692c404a5d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:15.426010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:15.426868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:15.427204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:15.427567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:15.428485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] RBAC: Adding query filter params (group.id=d3c7e9ff657443c59a02bf3c94b3416f, role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:15.428825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:15.430775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:15.447377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 183/367] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:15 2022] GET /identity/v3/role_assignments?group.id=d3c7e9ff657443c59a02bf3c94b3416f&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=2e43acd667e44e0bbba45926e652a64f => generated 260 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:15.447377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:15.586493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c895e2-bdc7-4508-a0be-42692c404a5d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:15.588893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:15.607616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bc5b3-aa96-4713-a570-d1123ee39002 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:15.635459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:15.636289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:15.636643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:15.637196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:15.638165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:15.638549 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:15.640339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:15.652455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 185/368] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:15.653386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:15.683641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6179b-ab91-4373-a05b-7c55d05b1d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:15.685111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6179b-ab91-4373-a05b-7c55d05b1d30 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:15.685582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6179b-ab91-4373-a05b-7c55d05b1d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:15.686042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6179b-ab91-4373-a05b-7c55d05b1d30 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:15.689015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6179b-ab91-4373-a05b-7c55d05b1d30 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:15.705414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6179b-ab91-4373-a05b-7c55d05b1d30 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:15.705414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 184/369] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:15 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:15.705414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6179b-ab91-4373-a05b-7c55d05b1d30 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:15.748073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:15.836673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6057d4-597f-4181-92cc-119093825eb4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:15.907999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:15.909621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:15.910595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:15.911321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:15.912981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:15.913729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:15.914073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6179b-ab91-4373-a05b-7c55d05b1d30 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:15.916408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:15.931262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 186/370] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:15.933444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:15.979529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe4cc9-cfa4-43b2-9662-d5fef2f9b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:15.981367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe4cc9-cfa4-43b2-9662-d5fef2f9b067 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:15.981907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe4cc9-cfa4-43b2-9662-d5fef2f9b067 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:15.982631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe4cc9-cfa4-43b2-9662-d5fef2f9b067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:16.023387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:16.089041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe4cc9-cfa4-43b2-9662-d5fef2f9b067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:16.093844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe4cc9-cfa4-43b2-9662-d5fef2f9b067 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:16.133312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a397e7-8821-490d-ad31-5158faef907f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:16.189635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:16.215984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331407ee-7028-490e-a4c3-b8b1a3f253e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:16.218916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331407ee-7028-490e-a4c3-b8b1a3f253e0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:16.219698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331407ee-7028-490e-a4c3-b8b1a3f253e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:16.219969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331407ee-7028-490e-a4c3-b8b1a3f253e0 None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:16.222411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331407ee-7028-490e-a4c3-b8b1a3f253e0 None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:16.224815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331407ee-7028-490e-a4c3-b8b1a3f253e0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:16.229059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 187/371] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:15 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:16.230293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331407ee-7028-490e-a4c3-b8b1a3f253e0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:16.328196 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 185/372] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:16.330052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe4cc9-cfa4-43b2-9662-d5fef2f9b067 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:16.434141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331407ee-7028-490e-a4c3-b8b1a3f253e0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:16.443912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:16.501292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f36fc-b135-42a9-9c86-122f3b072930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:16.502210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f36fc-b135-42a9-9c86-122f3b072930 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:16.502642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f36fc-b135-42a9-9c86-122f3b072930 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:16.503094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f36fc-b135-42a9-9c86-122f3b072930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:16.527446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe4cc9-cfa4-43b2-9662-d5fef2f9b067 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:16.581428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:16.589375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:16.590356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:16.594270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:16.594270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:16.594270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] RBAC: Adding query filter params (group.id=d3c7e9ff657443c59a02bf3c94b3416f, role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=e432781ae1bf4d319a43f8749c32be0f) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:16.594270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:16.594270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:16.601636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f36fc-b135-42a9-9c86-122f3b072930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:16.602733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 186/373] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:16 2022] GET /identity/v3/role_assignments?group.id=d3c7e9ff657443c59a02bf3c94b3416f&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:16.603830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:16.603966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f36fc-b135-42a9-9c86-122f3b072930 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:16.885423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 188/374] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:16.891039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f36fc-b135-42a9-9c86-122f3b072930 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:16.900757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf8af-0533-4e33-846c-7c89a51cf551 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:16.952536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:16.974206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4f918-6114-4d52-a03b-8b0bdbd6b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:16.976232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4f918-6114-4d52-a03b-8b0bdbd6b546 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:16.977015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4f918-6114-4d52-a03b-8b0bdbd6b546 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:16.977795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4f918-6114-4d52-a03b-8b0bdbd6b546 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:16.980056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4f918-6114-4d52-a03b-8b0bdbd6b546 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:16.982434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4f918-6114-4d52-a03b-8b0bdbd6b546 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:16.987477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 187/375] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:16 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:16.994153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:16.997175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4f918-6114-4d52-a03b-8b0bdbd6b546 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:17.013905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f36fc-b135-42a9-9c86-122f3b072930 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:17.096633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890879d-116c-4db4-96a7-982f1c1c67ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:17.097511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890879d-116c-4db4-96a7-982f1c1c67ed None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:17.097798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890879d-116c-4db4-96a7-982f1c1c67ed None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:17.098077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890879d-116c-4db4-96a7-982f1c1c67ed None admin] PATH_INFO: `/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:17.099363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890879d-116c-4db4-96a7-982f1c1c67ed None admin] RBAC: Authorizing `identity:get_project(project_id=e12b5dbdf665489f80381a9cc1ab6bd0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:17.100886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890879d-116c-4db4-96a7-982f1c1c67ed None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:17.103269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 189/376] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:16 2022] GET /identity/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0 => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:17.108299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890879d-116c-4db4-96a7-982f1c1c67ed None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:17.190257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4f918-6114-4d52-a03b-8b0bdbd6b546 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:17.276513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e551e-8940-4e66-a0e4-1e6f2ca0763c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:17.278115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e551e-8940-4e66-a0e4-1e6f2ca0763c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:17.278621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e551e-8940-4e66-a0e4-1e6f2ca0763c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:17.279118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e551e-8940-4e66-a0e4-1e6f2ca0763c None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:17.281556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890879d-116c-4db4-96a7-982f1c1c67ed None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:17.284605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e551e-8940-4e66-a0e4-1e6f2ca0763c None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:17.288249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e551e-8940-4e66-a0e4-1e6f2ca0763c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:17.291667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 188/377] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:16 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:17.301712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e551e-8940-4e66-a0e4-1e6f2ca0763c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:17.322061 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b7e9b-b41d-439e-8d46-3e33244fbceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:17.322931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7e9b-b41d-439e-8d46-3e33244fbceb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:17.323217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7e9b-b41d-439e-8d46-3e33244fbceb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:17.323522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7e9b-b41d-439e-8d46-3e33244fbceb None admin] PATH_INFO: `/v3/groups/17b3baee52b5486a8847fdac431124f9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:17.329232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b7e9b-b41d-439e-8d46-3e33244fbceb None admin] RBAC: Authorizing `identity:get_group(group_id=17b3baee52b5486a8847fdac431124f9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:17.330699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b7e9b-b41d-439e-8d46-3e33244fbceb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:17.332876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 190/378] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:17 2022] GET /identity/v3/groups/17b3baee52b5486a8847fdac431124f9 => generated 229 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:17.344310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b7e9b-b41d-439e-8d46-3e33244fbceb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:17.382845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:17.434268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:17.462141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b7e9b-b41d-439e-8d46-3e33244fbceb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:17.501396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a2c71-9548-40fb-abd6-21501f5b6988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:17.502390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a2c71-9548-40fb-abd6-21501f5b6988 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:17.502870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a2c71-9548-40fb-abd6-21501f5b6988 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:17.503349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a2c71-9548-40fb-abd6-21501f5b6988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:17.504457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a2c71-9548-40fb-abd6-21501f5b6988 None admin] RBAC: Adding query filter params (group.id=17b3baee52b5486a8847fdac431124f9, role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=e12b5dbdf665489f80381a9cc1ab6bd0) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:17.504918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a2c71-9548-40fb-abd6-21501f5b6988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:17.506590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a2c71-9548-40fb-abd6-21501f5b6988 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:17.513440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 191/379] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:17 2022] GET /identity/v3/role_assignments?group.id=17b3baee52b5486a8847fdac431124f9&scope.project.id=e12b5dbdf665489f80381a9cc1ab6bd0&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:17.515301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9b7f0-ad0b-4e67-ae90-b0137f6078c9 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:17.515557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e551e-8940-4e66-a0e4-1e6f2ca0763c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:17.516247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9b7f0-ad0b-4e67-ae90-b0137f6078c9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:17.516247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9b7f0-ad0b-4e67-ae90-b0137f6078c9 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:17.517840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 192/380] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:17.525057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076125b-26bd-4196-8e1a-6f6ac20124b2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:17.525525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076125b-26bd-4196-8e1a-6f6ac20124b2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:17.525906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076125b-26bd-4196-8e1a-6f6ac20124b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:17.545563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8076125b-26bd-4196-8e1a-6f6ac20124b2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:17.547055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8076125b-26bd-4196-8e1a-6f6ac20124b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:17.581450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:17.582644 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:17.583174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:17.583632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:17.585624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] RBAC: Adding query filter params (group.id=d3c7e9ff657443c59a02bf3c94b3416f, role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:17.586490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:17.589586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:17.607405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 189/381] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:17 2022] GET /identity/v3/role_assignments?group.id=d3c7e9ff657443c59a02bf3c94b3416f&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 260 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:17.630894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 193/382] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:17.654397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:17.835605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e604d-1224-413a-8c9f-4ea9ceb94f68 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:17.873478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:17.885868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad4a7-e1a2-49be-849c-ea027c6c5683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:17.886691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad4a7-e1a2-49be-849c-ea027c6c5683 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:17.886975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad4a7-e1a2-49be-849c-ea027c6c5683 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:17.887271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad4a7-e1a2-49be-849c-ea027c6c5683 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:17.900647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ad4a7-e1a2-49be-849c-ea027c6c5683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b57ad4a7-e1a2-49be-849c-ea027c6c5683 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:17.909709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:17.910820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:17.911831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:17.915320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 190/383] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:43:17.921176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076125b-26bd-4196-8e1a-6f6ac20124b2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:17.945762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:18.039165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076125b-26bd-4196-8e1a-6f6ac20124b2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:18.081380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd4a61-ebf3-4a0d-916c-b9ebba44cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:18.082431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4a61-ebf3-4a0d-916c-b9ebba44cf7f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:18.082938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4a61-ebf3-4a0d-916c-b9ebba44cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:18.083463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4a61-ebf3-4a0d-916c-b9ebba44cf7f None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:18.084461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd4a61-ebf3-4a0d-916c-b9ebba44cf7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:18.084984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd4a61-ebf3-4a0d-916c-b9ebba44cf7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:18.086579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd4a61-ebf3-4a0d-916c-b9ebba44cf7f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:18.096499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 194/384] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:18.104266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad4a7-e1a2-49be-849c-ea027c6c5683 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:18.303384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a36d-d59d-43f9-bd34-e4ab72b4080b None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:18.304159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a36d-d59d-43f9-bd34-e4ab72b4080b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:18.304646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a36d-d59d-43f9-bd34-e4ab72b4080b None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:18.306982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 195/385] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:18.310997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad4a7-e1a2-49be-849c-ea027c6c5683 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:18.312803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c1f1-bfad-4362-8672-46469eda20f4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:18.313349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c1f1-bfad-4362-8672-46469eda20f4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:18.313938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c1f1-bfad-4362-8672-46469eda20f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:18.355719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9e13c1f1-bfad-4362-8672-46469eda20f4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:18.357458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9e13c1f1-bfad-4362-8672-46469eda20f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:18.377682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f2558-17a5-4330-a383-4b567b576395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:18.379472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f2558-17a5-4330-a383-4b567b576395 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:18.380402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f2558-17a5-4330-a383-4b567b576395 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:18.381132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f2558-17a5-4330-a383-4b567b576395 None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:18.383886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f2558-17a5-4330-a383-4b567b576395 None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:18.386924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f2558-17a5-4330-a383-4b567b576395 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:18.390908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 191/386] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:18 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:18.399491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f2558-17a5-4330-a383-4b567b576395 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:18.456407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 196/387] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:18.484129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13c1f1-bfad-4362-8672-46469eda20f4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:18.507149 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:18.523670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:18.601395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f2558-17a5-4330-a383-4b567b576395 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:18.668625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df32ff4-ca72-435b-83b6-a0e691179413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:18.670534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df32ff4-ca72-435b-83b6-a0e691179413 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:18.671299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df32ff4-ca72-435b-83b6-a0e691179413 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:18.672027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df32ff4-ca72-435b-83b6-a0e691179413 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:18.674720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df32ff4-ca72-435b-83b6-a0e691179413 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:18.678710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df32ff4-ca72-435b-83b6-a0e691179413 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:18.684939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 192/388] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:18 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:18.696404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df32ff4-ca72-435b-83b6-a0e691179413 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:18.729581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13c1f1-bfad-4362-8672-46469eda20f4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:18.816993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633596-de43-4ed8-b85f-3a7f4d1c8da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:18.819779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633596-de43-4ed8-b85f-3a7f4d1c8da2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:18.820612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633596-de43-4ed8-b85f-3a7f4d1c8da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:18.821386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633596-de43-4ed8-b85f-3a7f4d1c8da2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:18.843480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f633596-de43-4ed8-b85f-3a7f4d1c8da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3f633596-de43-4ed8-b85f-3a7f4d1c8da2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:18.864341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:18.866687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:18.868358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:18.874018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 197/389] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:43:18.880895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633596-de43-4ed8-b85f-3a7f4d1c8da2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:19.023965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:19.041646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df32ff4-ca72-435b-83b6-a0e691179413 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:19.108191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:19.123748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633596-de43-4ed8-b85f-3a7f4d1c8da2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:19.135193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289ed33-c762-4652-af62-8c69be019cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:19.137124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289ed33-c762-4652-af62-8c69be019cfd None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:19.137895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289ed33-c762-4652-af62-8c69be019cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:19.138974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289ed33-c762-4652-af62-8c69be019cfd None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f/groups/d3c7e9ff657443c59a02bf3c94b3416f/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:19.143138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289ed33-c762-4652-af62-8c69be019cfd None admin] RBAC: Authorizing `identity:create_grant(project_id=e432781ae1bf4d319a43f8749c32be0f, group_id=d3c7e9ff657443c59a02bf3c94b3416f, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:19.146332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289ed33-c762-4652-af62-8c69be019cfd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:19.188909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf26ef9-9829-498e-9fc4-f6df189d3e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:19.190057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf26ef9-9829-498e-9fc4-f6df189d3e83 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:19.190521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf26ef9-9829-498e-9fc4-f6df189d3e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:19.191025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf26ef9-9829-498e-9fc4-f6df189d3e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:19.192539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf26ef9-9829-498e-9fc4-f6df189d3e83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:19.193319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf26ef9-9829-498e-9fc4-f6df189d3e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:19.196113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf26ef9-9829-498e-9fc4-f6df189d3e83 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:19.213311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 198/390] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:19.215199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7698e-07e9-4bb2-b039-ce0a8e4b54fe None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:19.215524 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7698e-07e9-4bb2-b039-ce0a8e4b54fe None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:19.215799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7698e-07e9-4bb2-b039-ce0a8e4b54fe None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:19.217564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 199/391] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:19.222359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7698e-07e9-4bb2-b039-ce0a8e4b54fe None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:19.225682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 193/392] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 1 03:43:18 2022] PUT /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f/groups/d3c7e9ff657443c59a02bf3c94b3416f/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:19.228982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb415-ad44-4ab7-b52e-976fc9c14ea0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:19.229842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb415-ad44-4ab7-b52e-976fc9c14ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:19.230666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb415-ad44-4ab7-b52e-976fc9c14ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:19.293685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0cafb415-ad44-4ab7-b52e-976fc9c14ea0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:19.296420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0cafb415-ad44-4ab7-b52e-976fc9c14ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:19.359653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7698e-07e9-4bb2-b039-ce0a8e4b54fe None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:19.388637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:19.396673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8ce64-a9c9-4db7-abef-9d674eeeb5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:19.398038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ce64-a9c9-4db7-abef-9d674eeeb5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:19.407258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ce64-a9c9-4db7-abef-9d674eeeb5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:19.407258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ce64-a9c9-4db7-abef-9d674eeeb5f8 None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:19.407258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8ce64-a9c9-4db7-abef-9d674eeeb5f8 None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:19.407258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8ce64-a9c9-4db7-abef-9d674eeeb5f8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:19.407258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 200/393] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:19 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:19.409193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8ce64-a9c9-4db7-abef-9d674eeeb5f8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:19.527604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 194/394] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:19.532703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8ce64-a9c9-4db7-abef-9d674eeeb5f8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:19.596731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e9817-ce2c-4ef3-b1af-266ef4b8eec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:19.597468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafb415-ad44-4ab7-b52e-976fc9c14ea0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:19.597773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9817-ce2c-4ef3-b1af-266ef4b8eec7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:19.598032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9817-ce2c-4ef3-b1af-266ef4b8eec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:19.598304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9817-ce2c-4ef3-b1af-266ef4b8eec7 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:19.599657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e9817-ce2c-4ef3-b1af-266ef4b8eec7 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:19.601181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e9817-ce2c-4ef3-b1af-266ef4b8eec7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:19.603313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 201/395] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:19 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:19.609299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e9817-ce2c-4ef3-b1af-266ef4b8eec7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:19.696315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:19.738743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e9817-ce2c-4ef3-b1af-266ef4b8eec7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:19.797406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e227c5-8837-4c23-84e6-5e4f3fa328c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:19.799794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e227c5-8837-4c23-84e6-5e4f3fa328c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:19.800654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e227c5-8837-4c23-84e6-5e4f3fa328c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:19.801632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e227c5-8837-4c23-84e6-5e4f3fa328c9 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/groups/d3c7e9ff657443c59a02bf3c94b3416f/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:19.816260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e227c5-8837-4c23-84e6-5e4f3fa328c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc9f58284ad54c079cfbd7d29dbf50ac, group_id=d3c7e9ff657443c59a02bf3c94b3416f, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:19.817832 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e227c5-8837-4c23-84e6-5e4f3fa328c9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:19.828996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafb415-ad44-4ab7-b52e-976fc9c14ea0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:19.842772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:19.870106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460410-cf2e-4b1e-af8d-60b8ee2e177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:19.870989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460410-cf2e-4b1e-af8d-60b8ee2e177c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:19.871296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460410-cf2e-4b1e-af8d-60b8ee2e177c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:19.871617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460410-cf2e-4b1e-af8d-60b8ee2e177c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:19.884925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f460410-cf2e-4b1e-af8d-60b8ee2e177c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7f460410-cf2e-4b1e-af8d-60b8ee2e177c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:19.893536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:19.895562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:19.897508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 195/396] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:43:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:19.901365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460410-cf2e-4b1e-af8d-60b8ee2e177c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:19.922754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 202/397] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 1 03:43:19 2022] PUT /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/groups/d3c7e9ff657443c59a02bf3c94b3416f/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:20.029228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460410-cf2e-4b1e-af8d-60b8ee2e177c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:20.069856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:20.070541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:20.070868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:20.071148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:20.072051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:20.072361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:20.073795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:20.084402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 196/398] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:20.092379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e227c5-8837-4c23-84e6-5e4f3fa328c9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:20.282706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e227c5-8837-4c23-84e6-5e4f3fa328c9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:20.351008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:20.375055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cc299-2716-4344-839d-a7c410b8598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:20.376692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cc299-2716-4344-839d-a7c410b8598a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:20.377191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cc299-2716-4344-839d-a7c410b8598a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:20.377697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cc299-2716-4344-839d-a7c410b8598a None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:20.383793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cc299-2716-4344-839d-a7c410b8598a None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:20.386706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cc299-2716-4344-839d-a7c410b8598a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:20.390588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 203/399] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:20 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:20.395939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:20.401236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd6c5-e782-4345-bf66-51827f96bc1a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:20.401733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd6c5-e782-4345-bf66-51827f96bc1a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:20.402234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd6c5-e782-4345-bf66-51827f96bc1a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:20.405391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 204/400] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:20.414758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d226d-3eb9-4d81-b3b1-34435afdcd3a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:20.415341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d226d-3eb9-4d81-b3b1-34435afdcd3a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:20.416040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d226d-3eb9-4d81-b3b1-34435afdcd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:20.453761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:20.473238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e08d226d-3eb9-4d81-b3b1-34435afdcd3a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:20.476271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e08d226d-3eb9-4d81-b3b1-34435afdcd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:20.544277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880723-e153-4b51-8fc8-c76157186a49 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:20.582511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcdc2c-f857-47c3-9dbf-fd1e013cc350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:20.583516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcdc2c-f857-47c3-9dbf-fd1e013cc350 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:20.583796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcdc2c-f857-47c3-9dbf-fd1e013cc350 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:20.584836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcdc2c-f857-47c3-9dbf-fd1e013cc350 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:20.586170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcdc2c-f857-47c3-9dbf-fd1e013cc350 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:20.587651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcdc2c-f857-47c3-9dbf-fd1e013cc350 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:20.589838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 197/401] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:20 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:20.595751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcdc2c-f857-47c3-9dbf-fd1e013cc350 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:20.610083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 205/402] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:20.660352 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d226d-3eb9-4d81-b3b1-34435afdcd3a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:20.747253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcdc2c-f857-47c3-9dbf-fd1e013cc350 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:20.749069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:20.795106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956db50d-d58e-41ed-b3c9-3605ad0d06ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:20.797971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db50d-d58e-41ed-b3c9-3605ad0d06ff None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:20.798458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db50d-d58e-41ed-b3c9-3605ad0d06ff None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:20.798951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db50d-d58e-41ed-b3c9-3605ad0d06ff None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f/groups/d3c7e9ff657443c59a02bf3c94b3416f/roles/a3df9ed8088248bbae25abcf9937fa11` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:20.808428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956db50d-d58e-41ed-b3c9-3605ad0d06ff None admin] RBAC: Authorizing `identity:create_grant(project_id=e432781ae1bf4d319a43f8749c32be0f, group_id=d3c7e9ff657443c59a02bf3c94b3416f, role_id=a3df9ed8088248bbae25abcf9937fa11)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:20.811061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956db50d-d58e-41ed-b3c9-3605ad0d06ff None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:20.928526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d226d-3eb9-4d81-b3b1-34435afdcd3a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:20.938645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 198/403] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 1 03:43:20 2022] PUT /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f/groups/d3c7e9ff657443c59a02bf3c94b3416f/roles/a3df9ed8088248bbae25abcf9937fa11 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:20.940074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce85c6-2775-4ea3-83ce-68d89e6a7b25 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:20.940364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce85c6-2775-4ea3-83ce-68d89e6a7b25 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:20.940669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce85c6-2775-4ea3-83ce-68d89e6a7b25 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:20.941978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 199/404] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:20.958365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e92b-3799-4596-8926-b15719a8ceef None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:20.958628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e92b-3799-4596-8926-b15719a8ceef None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:20.958916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e92b-3799-4596-8926-b15719a8ceef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:20.989571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6672e92b-3799-4596-8926-b15719a8ceef None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:20.990896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6672e92b-3799-4596-8926-b15719a8ceef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:21.012525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:21.060282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc579d-17f6-448c-8164-449798310c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:21.061019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc579d-17f6-448c-8164-449798310c5a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:21.061303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc579d-17f6-448c-8164-449798310c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:21.061582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc579d-17f6-448c-8164-449798310c5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:21.098694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc579d-17f6-448c-8164-449798310c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-42fc579d-17f6-448c-8164-449798310c5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:21.106889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:21.110270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:21.113344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:21.121298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 206/405] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:43:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:21.128318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc579d-17f6-448c-8164-449798310c5a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:21.173103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 200/406] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:21.228429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672e92b-3799-4596-8926-b15719a8ceef None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:21.280645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc579d-17f6-448c-8164-449798310c5a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:21.325273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:21.325864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:21.326147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:21.326406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:21.327208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:21.331274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:21.331274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:21.346843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 207/407] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:21.353483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:21.363620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:21.375053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672e92b-3799-4596-8926-b15719a8ceef None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:21.416442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1c28-4a99-4de8-b64c-e6aac927d79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:21.417860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1c28-4a99-4de8-b64c-e6aac927d79e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:21.418383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1c28-4a99-4de8-b64c-e6aac927d79e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:21.418860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1c28-4a99-4de8-b64c-e6aac927d79e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:21.432764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a1c28-4a99-4de8-b64c-e6aac927d79e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-af5a1c28-4a99-4de8-b64c-e6aac927d79e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:21.441315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:21.442265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:43:21.443035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:21.445934 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 201/408] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:43:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:21.454524 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1c28-4a99-4de8-b64c-e6aac927d79e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:21.461101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:21.569356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1c28-4a99-4de8-b64c-e6aac927d79e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:21.607267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:21.608297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:21.608770 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:21.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:21.610198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:21.610695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:21.612331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:21.622260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 202/409] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:21.637754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:21.665389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321f1e-47b5-4cce-9bcc-34ecc3238be7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:21.697707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:21.746964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53165f-bc91-4450-91d3-2fafb896971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:21.748441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53165f-bc91-4450-91d3-2fafb896971f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:21.748923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53165f-bc91-4450-91d3-2fafb896971f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:21.749410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53165f-bc91-4450-91d3-2fafb896971f None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:21.753030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53165f-bc91-4450-91d3-2fafb896971f None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:21.755680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53165f-bc91-4450-91d3-2fafb896971f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:21.773079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 208/410] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:21 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:21.783959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53165f-bc91-4450-91d3-2fafb896971f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:21.853594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10189fc-048d-4cb4-a05c-8ac92e414e67 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:21.893660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ab583-2846-4154-987b-a90c2a91d844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:21.894856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ab583-2846-4154-987b-a90c2a91d844 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:21.895464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ab583-2846-4154-987b-a90c2a91d844 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:21.895899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ab583-2846-4154-987b-a90c2a91d844 None admin] PATH_INFO: `/v3/groups/17b3baee52b5486a8847fdac431124f9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:21.897908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ab583-2846-4154-987b-a90c2a91d844 None admin] RBAC: Authorizing `identity:get_group(group_id=17b3baee52b5486a8847fdac431124f9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:21.899582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ab583-2846-4154-987b-a90c2a91d844 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:21.901848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 203/411] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:21 2022] GET /identity/v3/groups/17b3baee52b5486a8847fdac431124f9 => generated 229 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:21.920115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ab583-2846-4154-987b-a90c2a91d844 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:22.016698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:22.063480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ab583-2846-4154-987b-a90c2a91d844 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:22.099268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53165f-bc91-4450-91d3-2fafb896971f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:22.111631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:22.131011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40b346-6550-46e4-a63b-e995c8067e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:22.133520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40b346-6550-46e4-a63b-e995c8067e54 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:22.134737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40b346-6550-46e4-a63b-e995c8067e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:22.135286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40b346-6550-46e4-a63b-e995c8067e54 None admin] PATH_INFO: `/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:22.137656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40b346-6550-46e4-a63b-e995c8067e54 None admin] RBAC: Authorizing `identity:get_project(project_id=e12b5dbdf665489f80381a9cc1ab6bd0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:22.140528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40b346-6550-46e4-a63b-e995c8067e54 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:22.143514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 204/412] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:21 2022] GET /identity/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:22.154531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40b346-6550-46e4-a63b-e995c8067e54 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:22.160058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efae7f-0a8a-48b0-8317-cf826c36a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:22.162241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efae7f-0a8a-48b0-8317-cf826c36a4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:22.163287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efae7f-0a8a-48b0-8317-cf826c36a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:22.163776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efae7f-0a8a-48b0-8317-cf826c36a4b7 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:22.168062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0efae7f-0a8a-48b0-8317-cf826c36a4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:22.169812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0efae7f-0a8a-48b0-8317-cf826c36a4b7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:22.172801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 209/413] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:21 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:22.180256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efae7f-0a8a-48b0-8317-cf826c36a4b7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:22.378076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40b346-6550-46e4-a63b-e995c8067e54 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:22.446405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ac983-17d5-446d-93f8-1af1eca62521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:22.447932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ac983-17d5-446d-93f8-1af1eca62521 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:22.448442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ac983-17d5-446d-93f8-1af1eca62521 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:22.449166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ac983-17d5-446d-93f8-1af1eca62521 None admin] PATH_INFO: `/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0/groups/17b3baee52b5486a8847fdac431124f9/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:22.453111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ac983-17d5-446d-93f8-1af1eca62521 None admin] RBAC: Authorizing `identity:create_grant(project_id=e12b5dbdf665489f80381a9cc1ab6bd0, group_id=17b3baee52b5486a8847fdac431124f9, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:22.456120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ac983-17d5-446d-93f8-1af1eca62521 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:22.491019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efae7f-0a8a-48b0-8317-cf826c36a4b7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:22.542037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 205/414] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 1 03:43:22 2022] PUT /identity/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0/groups/17b3baee52b5486a8847fdac431124f9/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:22.574186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe01ba7-ff47-46c5-b58f-b6d8af26371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:22.576739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe01ba7-ff47-46c5-b58f-b6d8af26371b None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:22.577698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe01ba7-ff47-46c5-b58f-b6d8af26371b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:22.578400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe01ba7-ff47-46c5-b58f-b6d8af26371b None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/groups/d3c7e9ff657443c59a02bf3c94b3416f/roles/a3df9ed8088248bbae25abcf9937fa11` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:22.583513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe01ba7-ff47-46c5-b58f-b6d8af26371b None admin] RBAC: Authorizing `identity:create_grant(project_id=dc9f58284ad54c079cfbd7d29dbf50ac, group_id=d3c7e9ff657443c59a02bf3c94b3416f, role_id=a3df9ed8088248bbae25abcf9937fa11)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:22.587438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe01ba7-ff47-46c5-b58f-b6d8af26371b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:22.685151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 210/415] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 1 03:43:22 2022] PUT /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac/groups/d3c7e9ff657443c59a02bf3c94b3416f/roles/a3df9ed8088248bbae25abcf9937fa11 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:22.997577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6c8d-2bd0-4a09-a398-8dd9d9be873b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:22.998026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6c8d-2bd0-4a09-a398-8dd9d9be873b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:22.998200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6c8d-2bd0-4a09-a398-8dd9d9be873b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:22.999831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 206/416] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:23.005650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27dcbc-963f-45b0-b167-2073b7b784cc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:23.006608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27dcbc-963f-45b0-b167-2073b7b784cc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:23.006761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27dcbc-963f-45b0-b167-2073b7b784cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:23.012935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0f5c4-5f8c-4345-8713-7f9d6a15bba5 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:23.013229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0f5c4-5f8c-4345-8713-7f9d6a15bba5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:23.013475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0f5c4-5f8c-4345-8713-7f9d6a15bba5 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:23.014996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 207/417] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:23.020676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23fd80-d318-48b3-a7aa-6332ce2e62ce None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:23.021269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23fd80-d318-48b3-a7aa-6332ce2e62ce None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:23.021269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23fd80-d318-48b3-a7aa-6332ce2e62ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:23.046152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2c27dcbc-963f-45b0-b167-2073b7b784cc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:23.047393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2c27dcbc-963f-45b0-b167-2073b7b784cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:23.050495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2b23fd80-d318-48b3-a7aa-6332ce2e62ce None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:23.051701 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2b23fd80-d318-48b3-a7aa-6332ce2e62ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:23.056164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:23.169979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:23.238390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 208/418] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:23.274326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 211/419] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:23.282359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23fd80-d318-48b3-a7aa-6332ce2e62ce None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:23.299783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27dcbc-963f-45b0-b167-2073b7b784cc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:23.486268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27dcbc-963f-45b0-b167-2073b7b784cc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:23.527583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f78a25-03bc-417e-94dd-d97072378da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:23.528787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f78a25-03bc-417e-94dd-d97072378da9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:23.529297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f78a25-03bc-417e-94dd-d97072378da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:23.529859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f78a25-03bc-417e-94dd-d97072378da9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:23.544907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f78a25-03bc-417e-94dd-d97072378da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-20f78a25-03bc-417e-94dd-d97072378da9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:23.553940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:23.555161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:23.555975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:23.559578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23fd80-d318-48b3-a7aa-6332ce2e62ce None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:23.569478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 212/420] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:23.570788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f78a25-03bc-417e-94dd-d97072378da9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:23.631683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52c91d-c760-4ba0-a62a-6e691309ce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:23.632885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52c91d-c760-4ba0-a62a-6e691309ce3f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:23.633365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52c91d-c760-4ba0-a62a-6e691309ce3f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:23.633852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52c91d-c760-4ba0-a62a-6e691309ce3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:23.639610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:23.658904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:23.661461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52c91d-c760-4ba0-a62a-6e691309ce3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-9e52c91d-c760-4ba0-a62a-6e691309ce3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:23.669743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:23.671543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:23.673099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 209/421] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:23.684564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52c91d-c760-4ba0-a62a-6e691309ce3f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:23.805643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f78a25-03bc-417e-94dd-d97072378da9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:23.867073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:23.867792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:23.868070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:23.868344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:23.869113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:23.869421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:23.870875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:23.883938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 213/422] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:23.887136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:23.949618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52c91d-c760-4ba0-a62a-6e691309ce3f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:23.996093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:24.002007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccdf46-95ae-4647-aa66-a6a02766c8e3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:24.008340 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29544215-11fa-44ef-897b-5972f83524c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:24.010183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29544215-11fa-44ef-897b-5972f83524c8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:24.011775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29544215-11fa-44ef-897b-5972f83524c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:24.012518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29544215-11fa-44ef-897b-5972f83524c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:24.017641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29544215-11fa-44ef-897b-5972f83524c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:24.017641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29544215-11fa-44ef-897b-5972f83524c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:24.021059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29544215-11fa-44ef-897b-5972f83524c8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:24.047260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 210/423] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:43:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:24.047260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df7c1d-6f8b-4ef4-bdc3-fb2518b1eade None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:24.047260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df7c1d-6f8b-4ef4-bdc3-fb2518b1eade None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:24.047260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df7c1d-6f8b-4ef4-bdc3-fb2518b1eade None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:24.047260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 211/424] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:24.050023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3646e5-0e5c-4614-bac6-12a42d301ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:24.050987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3646e5-0e5c-4614-bac6-12a42d301ccc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:24.052266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df7c1d-6f8b-4ef4-bdc3-fb2518b1eade None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:24.052727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3646e5-0e5c-4614-bac6-12a42d301ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:24.053021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3646e5-0e5c-4614-bac6-12a42d301ccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:24.123462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:24.176876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3646e5-0e5c-4614-bac6-12a42d301ccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:24.178952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3646e5-0e5c-4614-bac6-12a42d301ccc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:24.204564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df7c1d-6f8b-4ef4-bdc3-fb2518b1eade None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:24.288785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e627b-73ed-47fb-add7-081dc05a4638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:24.290775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e627b-73ed-47fb-add7-081dc05a4638 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:24.291703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e627b-73ed-47fb-add7-081dc05a4638 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:24.293681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e627b-73ed-47fb-add7-081dc05a4638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:24.329702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:24.348997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 214/425] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:24.354280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67051d76-c3cd-471b-9d85-31a069a8c980 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:24.356422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67051d76-c3cd-471b-9d85-31a069a8c980 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:24.357265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67051d76-c3cd-471b-9d85-31a069a8c980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:24.394123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-67051d76-c3cd-471b-9d85-31a069a8c980 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:24.395375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-67051d76-c3cd-471b-9d85-31a069a8c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:24.431588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e627b-73ed-47fb-add7-081dc05a4638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:24.435810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e627b-73ed-47fb-add7-081dc05a4638 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:24.490014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 215/426] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:24.491421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67051d76-c3cd-471b-9d85-31a069a8c980 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:24.547130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:24.663194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 212/427] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:24.663194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e627b-73ed-47fb-add7-081dc05a4638 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:24.703952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67051d76-c3cd-471b-9d85-31a069a8c980 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:24.757023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:24.778209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcde140-f58b-4fd9-a732-0ce6cb054351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:24.780639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcde140-f58b-4fd9-a732-0ce6cb054351 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:24.781440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcde140-f58b-4fd9-a732-0ce6cb054351 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:24.782190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcde140-f58b-4fd9-a732-0ce6cb054351 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:24.785110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcde140-f58b-4fd9-a732-0ce6cb054351 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:24.788458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcde140-f58b-4fd9-a732-0ce6cb054351 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:24.792597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 216/428] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:24 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:24.794336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcde140-f58b-4fd9-a732-0ce6cb054351 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:24.889091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e627b-73ed-47fb-add7-081dc05a4638 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:24.958028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892338e0-de0e-4a96-a6e7-21abdbe91f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:24.959971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892338e0-de0e-4a96-a6e7-21abdbe91f83 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:24.960610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892338e0-de0e-4a96-a6e7-21abdbe91f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:24.961141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892338e0-de0e-4a96-a6e7-21abdbe91f83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:24.976992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892338e0-de0e-4a96-a6e7-21abdbe91f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-892338e0-de0e-4a96-a6e7-21abdbe91f83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:24.988361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:24.989642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:24.991523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:24.995046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 213/429] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:43:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:24.996259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892338e0-de0e-4a96-a6e7-21abdbe91f83 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:25.020382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcde140-f58b-4fd9-a732-0ce6cb054351 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:25.044887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:25.093236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c796-53b5-4262-9442-49ad99dbd680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:25.095102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c796-53b5-4262-9442-49ad99dbd680 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.096152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c796-53b5-4262-9442-49ad99dbd680 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.096898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c796-53b5-4262-9442-49ad99dbd680 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.099440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c796-53b5-4262-9442-49ad99dbd680 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:25.102985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c796-53b5-4262-9442-49ad99dbd680 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:25.106471 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 217/430] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:24 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:25.113234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c796-53b5-4262-9442-49ad99dbd680 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:25.238826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892338e0-de0e-4a96-a6e7-21abdbe91f83 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:25.288046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:25.312952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c796-53b5-4262-9442-49ad99dbd680 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:25.318850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614a2ee-0739-481f-91b5-9975a263e238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:25.320854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614a2ee-0739-481f-91b5-9975a263e238 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.321516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614a2ee-0739-481f-91b5-9975a263e238 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.322191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614a2ee-0739-481f-91b5-9975a263e238 None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.324783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614a2ee-0739-481f-91b5-9975a263e238 None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:25.327819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614a2ee-0739-481f-91b5-9975a263e238 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:25.331282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 214/431] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:24 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:25.332877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614a2ee-0739-481f-91b5-9975a263e238 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:25.393940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:25.395519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.396007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.397043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.398488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:25.399153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:25.403402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:25.414874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 218/432] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:25.416106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:25.487708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:25.511382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614a2ee-0739-481f-91b5-9975a263e238 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:25.565938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccb717-cd03-43f1-8ab5-259dd3585a85 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:25.576286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7f08c-b38e-409b-904f-8fabddf0f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:25.577655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7f08c-b38e-409b-904f-8fabddf0f6aa None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.578163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7f08c-b38e-409b-904f-8fabddf0f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.578703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7f08c-b38e-409b-904f-8fabddf0f6aa None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.580807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7f08c-b38e-409b-904f-8fabddf0f6aa None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:25.583335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7f08c-b38e-409b-904f-8fabddf0f6aa None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:25.587199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 215/433] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:25 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:25.589321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509c43-30e8-48be-ba6b-9d998c6feaee None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.589719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509c43-30e8-48be-ba6b-9d998c6feaee None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.590221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509c43-30e8-48be-ba6b-9d998c6feaee None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.591164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:25.592907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 216/434] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:25.594136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509c43-30e8-48be-ba6b-9d998c6feaee None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:25.604201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700cc25-3e39-406b-b166-61b644e0dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:25.605503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700cc25-3e39-406b-b166-61b644e0dedc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.605788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700cc25-3e39-406b-b166-61b644e0dedc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.606058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700cc25-3e39-406b-b166-61b644e0dedc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.606894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700cc25-3e39-406b-b166-61b644e0dedc None admin] RBAC: Adding query filter params (group.id=d3c7e9ff657443c59a02bf3c94b3416f, role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=e432781ae1bf4d319a43f8749c32be0f) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:25.607201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700cc25-3e39-406b-b166-61b644e0dedc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:25.608722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700cc25-3e39-406b-b166-61b644e0dedc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:25.618336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 219/435] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:25 2022] GET /identity/v3/role_assignments?group.id=d3c7e9ff657443c59a02bf3c94b3416f&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=2e43acd667e44e0bbba45926e652a64f => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:25.620783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84ebb-0d90-4003-bd79-dafab090afec None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.621294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84ebb-0d90-4003-bd79-dafab090afec None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.621772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84ebb-0d90-4003-bd79-dafab090afec None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.624422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 220/436] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:25.625709 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa84ebb-0d90-4003-bd79-dafab090afec None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:25.744454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509c43-30e8-48be-ba6b-9d998c6feaee None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:25.798761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4cfd-9912-4143-a335-ff8510ad1ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:25.800388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4cfd-9912-4143-a335-ff8510ad1ecd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.800388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4cfd-9912-4143-a335-ff8510ad1ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.800604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa84ebb-0d90-4003-bd79-dafab090afec None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:25.800795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4cfd-9912-4143-a335-ff8510ad1ecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.841161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:25.852946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee0fb6-2636-42f5-8b11-3a160b54d0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:25.857025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee0fb6-2636-42f5-8b11-3a160b54d0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.857025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee0fb6-2636-42f5-8b11-3a160b54d0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.857025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee0fb6-2636-42f5-8b11-3a160b54d0a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.857025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee0fb6-2636-42f5-8b11-3a160b54d0a1 None admin] RBAC: Adding query filter params (group.id=d3c7e9ff657443c59a02bf3c94b3416f, role.id=2e43acd667e44e0bbba45926e652a64f, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:25.857025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee0fb6-2636-42f5-8b11-3a160b54d0a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:25.857025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee0fb6-2636-42f5-8b11-3a160b54d0a1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:25.864306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 221/437] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:25 2022] GET /identity/v3/role_assignments?group.id=d3c7e9ff657443c59a02bf3c94b3416f&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=2e43acd667e44e0bbba45926e652a64f => generated 611 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:25.866184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5a99-6bbd-4603-ab95-8ec5b3efb111 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:25.866473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5a99-6bbd-4603-ab95-8ec5b3efb111 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:25.866752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5a99-6bbd-4603-ab95-8ec5b3efb111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:25.893808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e51d5a99-6bbd-4603-ab95-8ec5b3efb111 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:25.895154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e51d5a99-6bbd-4603-ab95-8ec5b3efb111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:25.921801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4cfd-9912-4143-a335-ff8510ad1ecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:25.924110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4cfd-9912-4143-a335-ff8510ad1ecd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:25.974948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:26.015412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 222/438] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:26.017341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab03c7-1c58-4d08-923e-8f75b44685e8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.017761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab03c7-1c58-4d08-923e-8f75b44685e8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.018207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab03c7-1c58-4d08-923e-8f75b44685e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.038702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-40ab03c7-1c58-4d08-923e-8f75b44685e8 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:26.040224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-40ab03c7-1c58-4d08-923e-8f75b44685e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:26.061822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 217/439] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:26.062715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4cfd-9912-4143-a335-ff8510ad1ecd None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:26.162576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:26.164247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 223/440] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:26.165825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab03c7-1c58-4d08-923e-8f75b44685e8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:26.220859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4cfd-9912-4143-a335-ff8510ad1ecd None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:26.263335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f44844-1f85-4a8f-9f59-f179f7b2cc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:26.264224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f44844-1f85-4a8f-9f59-f179f7b2cc3a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.264685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f44844-1f85-4a8f-9f59-f179f7b2cc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.265029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f44844-1f85-4a8f-9f59-f179f7b2cc3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.278952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f44844-1f85-4a8f-9f59-f179f7b2cc3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-69f44844-1f85-4a8f-9f59-f179f7b2cc3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:26.289598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:26.291532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 218/441] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:26.293217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f44844-1f85-4a8f-9f59-f179f7b2cc3a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:26.296555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab03c7-1c58-4d08-923e-8f75b44685e8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:26.309251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:26.340558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83c15d-b06a-4259-a06e-46702cbcb124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:26.341335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83c15d-b06a-4259-a06e-46702cbcb124 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.341622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83c15d-b06a-4259-a06e-46702cbcb124 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.341907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83c15d-b06a-4259-a06e-46702cbcb124 None admin] PATH_INFO: `/v3/projects/e432781ae1bf4d319a43f8749c32be0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.343847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83c15d-b06a-4259-a06e-46702cbcb124 None admin] RBAC: Authorizing `identity:get_project(project_id=e432781ae1bf4d319a43f8749c32be0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:26.346031 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83c15d-b06a-4259-a06e-46702cbcb124 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:26.348554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 224/442] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/projects/e432781ae1bf4d319a43f8749c32be0f => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:26.350118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83c15d-b06a-4259-a06e-46702cbcb124 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:26.407872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f44844-1f85-4a8f-9f59-f179f7b2cc3a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:26.426009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:26.445231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2a60d-c421-4376-a147-aaf702f62333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:26.446027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2a60d-c421-4376-a147-aaf702f62333 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.446310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2a60d-c421-4376-a147-aaf702f62333 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.446586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2a60d-c421-4376-a147-aaf702f62333 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.458256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2a60d-c421-4376-a147-aaf702f62333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-50d2a60d-c421-4376-a147-aaf702f62333 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:26.467151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:26.468625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 219/443] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:26.470845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2a60d-c421-4376-a147-aaf702f62333 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:26.506584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83c15d-b06a-4259-a06e-46702cbcb124 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:26.549842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:26.550667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.551002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.551397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.552249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:26.552688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:26.554345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:26.564419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 225/444] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:26.565795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:26.611680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2a60d-c421-4376-a147-aaf702f62333 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:26.626874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:26.665742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d783eac-395f-4881-90ab-50eba5911967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:26.666872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d783eac-395f-4881-90ab-50eba5911967 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.667353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d783eac-395f-4881-90ab-50eba5911967 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.667801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d783eac-395f-4881-90ab-50eba5911967 None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.669833 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d783eac-395f-4881-90ab-50eba5911967 None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:26.672259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d783eac-395f-4881-90ab-50eba5911967 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:26.674921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 220/445] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:26.676079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d783eac-395f-4881-90ab-50eba5911967 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:26.715341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c516f-a263-4965-85dd-16df193c3b31 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:26.736804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:26.763216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:26.764126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.764534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.764981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.766289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:26.766763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:26.769135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:26.782788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 226/446] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:26.788336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:26.819499 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d783eac-395f-4881-90ab-50eba5911967 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:26.865825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ca5e-c129-4702-aecc-0cf3311fb785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:26.866586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ca5e-c129-4702-aecc-0cf3311fb785 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.866952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ca5e-c129-4702-aecc-0cf3311fb785 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.867417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ca5e-c129-4702-aecc-0cf3311fb785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.921079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fde1a-6c54-49fb-9c3f-006aa25ce3a0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:26.923835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:26.968394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4ca5e-c129-4702-aecc-0cf3311fb785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:26.970946 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4ca5e-c129-4702-aecc-0cf3311fb785 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:26.974119 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:26.975150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:26.975649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:26.975954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:26.976810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] RBAC: Adding query filter params (group.id=d3c7e9ff657443c59a02bf3c94b3416f, role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=e432781ae1bf4d319a43f8749c32be0f) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:26.977126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:26.978646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:26.986444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 227/447] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/role_assignments?group.id=d3c7e9ff657443c59a02bf3c94b3416f&scope.project.id=e432781ae1bf4d319a43f8749c32be0f&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:26.987664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:27.042589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:27.119404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 221/448] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:27.129169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ca5e-c129-4702-aecc-0cf3311fb785 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:27.165255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5b72-ff18-405d-af95-4b60832db636 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:27.229740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53186cd-f2eb-4bf1-97f7-1adf38eb7dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:27.230883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53186cd-f2eb-4bf1-97f7-1adf38eb7dbd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:27.232752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53186cd-f2eb-4bf1-97f7-1adf38eb7dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:27.233287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53186cd-f2eb-4bf1-97f7-1adf38eb7dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:27.291440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:27.296425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ca5e-c129-4702-aecc-0cf3311fb785 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:27.347407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53186cd-f2eb-4bf1-97f7-1adf38eb7dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:27.350134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53186cd-f2eb-4bf1-97f7-1adf38eb7dbd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:27.352876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01699d7b-95ba-44b3-a959-df40fcb2fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:27.354134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01699d7b-95ba-44b3-a959-df40fcb2fee8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:27.354586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01699d7b-95ba-44b3-a959-df40fcb2fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:27.355014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01699d7b-95ba-44b3-a959-df40fcb2fee8 None admin] PATH_INFO: `/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:27.357006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01699d7b-95ba-44b3-a959-df40fcb2fee8 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9f58284ad54c079cfbd7d29dbf50ac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:27.359640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01699d7b-95ba-44b3-a959-df40fcb2fee8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:27.362784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 222/449] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:27 2022] GET /identity/v3/projects/dc9f58284ad54c079cfbd7d29dbf50ac => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:27.369680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01699d7b-95ba-44b3-a959-df40fcb2fee8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:27.396928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:27.538773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 228/450] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:27.544884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53186cd-f2eb-4bf1-97f7-1adf38eb7dbd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:27.545095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01699d7b-95ba-44b3-a959-df40fcb2fee8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:27.604876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee390fb-480a-4265-9c44-8c217e5a0735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:27.606109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee390fb-480a-4265-9c44-8c217e5a0735 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:27.606801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee390fb-480a-4265-9c44-8c217e5a0735 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:27.607307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee390fb-480a-4265-9c44-8c217e5a0735 None admin] PATH_INFO: `/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:27.610601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee390fb-480a-4265-9c44-8c217e5a0735 None admin] RBAC: Authorizing `identity:get_group(group_id=d3c7e9ff657443c59a02bf3c94b3416f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:27.612374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee390fb-480a-4265-9c44-8c217e5a0735 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:27.614307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 223/451] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:27 2022] GET /identity/v3/groups/d3c7e9ff657443c59a02bf3c94b3416f => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:27.623355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee390fb-480a-4265-9c44-8c217e5a0735 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:27.643615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:27.708866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53186cd-f2eb-4bf1-97f7-1adf38eb7dbd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:27.746941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:27.768109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee390fb-480a-4265-9c44-8c217e5a0735 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:27.769620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873a0c2-2075-4d54-969e-4092b185cb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:27.771581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873a0c2-2075-4d54-969e-4092b185cb42 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:27.772056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873a0c2-2075-4d54-969e-4092b185cb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:27.772480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873a0c2-2075-4d54-969e-4092b185cb42 None admin] PATH_INFO: `/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:27.773952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873a0c2-2075-4d54-969e-4092b185cb42 None admin] RBAC: Authorizing `identity:get_project(project_id=e12b5dbdf665489f80381a9cc1ab6bd0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:27.775664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873a0c2-2075-4d54-969e-4092b185cb42 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:27.779030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 229/452] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:43:27 2022] GET /identity/v3/projects/e12b5dbdf665489f80381a9cc1ab6bd0 => generated 344 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:27.784865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873a0c2-2075-4d54-969e-4092b185cb42 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:27.813030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:27.813791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:27.814082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:27.814328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:27.815113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] RBAC: Adding query filter params (group.id=d3c7e9ff657443c59a02bf3c94b3416f, role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:27.815465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:27.816946 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:27.831496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 224/453] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:27 2022] GET /identity/v3/role_assignments?group.id=d3c7e9ff657443c59a02bf3c94b3416f&scope.project.id=dc9f58284ad54c079cfbd7d29dbf50ac&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:27.912330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873a0c2-2075-4d54-969e-4092b185cb42 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:27.953955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f453638-a3ba-4903-b3a9-bf7e8705bcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:27.954856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453638-a3ba-4903-b3a9-bf7e8705bcdd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:27.955140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453638-a3ba-4903-b3a9-bf7e8705bcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:27.955447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453638-a3ba-4903-b3a9-bf7e8705bcdd None admin] PATH_INFO: `/v3/groups/17b3baee52b5486a8847fdac431124f9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:27.956881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f453638-a3ba-4903-b3a9-bf7e8705bcdd None admin] RBAC: Authorizing `identity:get_group(group_id=17b3baee52b5486a8847fdac431124f9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:27.958377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f453638-a3ba-4903-b3a9-bf7e8705bcdd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:27.960255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 230/454] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 1 03:43:27 2022] GET /identity/v3/groups/17b3baee52b5486a8847fdac431124f9 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:27.966669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:28.084670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f502f7-0ba7-4e45-9ff3-43b4e33f900e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:28.133525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05488ed2-2f7b-47ee-9e2b-4894bddd277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:28.134250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488ed2-2f7b-47ee-9e2b-4894bddd277e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:28.134737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488ed2-2f7b-47ee-9e2b-4894bddd277e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:28.135039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488ed2-2f7b-47ee-9e2b-4894bddd277e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:28.136694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488ed2-2f7b-47ee-9e2b-4894bddd277e None admin] RBAC: Adding query filter params (group.id=17b3baee52b5486a8847fdac431124f9, role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=e12b5dbdf665489f80381a9cc1ab6bd0) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:28.137244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488ed2-2f7b-47ee-9e2b-4894bddd277e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:28.140141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488ed2-2f7b-47ee-9e2b-4894bddd277e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:28.152498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 225/455] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 1 03:43:27 2022] GET /identity/v3/role_assignments?group.id=17b3baee52b5486a8847fdac431124f9&scope.project.id=e12b5dbdf665489f80381a9cc1ab6bd0&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:30.753014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714cc30-9a41-4113-9afc-3a09671cddee None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:30.754019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714cc30-9a41-4113-9afc-3a09671cddee None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:30.754505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714cc30-9a41-4113-9afc-3a09671cddee None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:30.756653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 231/456] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:30.773685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6003414-4839-4379-a5d3-c50be7fc1e81 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:30.774685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6003414-4839-4379-a5d3-c50be7fc1e81 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:30.775173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6003414-4839-4379-a5d3-c50be7fc1e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:30.805001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f6003414-4839-4379-a5d3-c50be7fc1e81 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:30.806553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f6003414-4839-4379-a5d3-c50be7fc1e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:30.809040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:30.909947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 226/457] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:30.947766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714cc30-9a41-4113-9afc-3a09671cddee None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:30.951184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:31.057094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e883c3a-1e13-4e10-8f83-1313c516d901 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.057775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e883c3a-1e13-4e10-8f83-1313c516d901 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.058257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e883c3a-1e13-4e10-8f83-1313c516d901 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.061323 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 227/458] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:43:31.070345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48607f57-28ba-4ad7-87f2-a6a2b3e5c4d5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.070973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48607f57-28ba-4ad7-87f2-a6a2b3e5c4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.071506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48607f57-28ba-4ad7-87f2-a6a2b3e5c4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.113882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-48607f57-28ba-4ad7-87f2-a6a2b3e5c4d5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:31.116426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-48607f57-28ba-4ad7-87f2-a6a2b3e5c4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:31.125528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714cc30-9a41-4113-9afc-3a09671cddee None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:31.236306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:31.237586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.238056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.238525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.266045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:31.267688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:31.287421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 228/459] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:31.295304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:31.296261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:31.297144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:31.297144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:31.297144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:31.297144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:31.297144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 232/460] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:31.305016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48607f57-28ba-4ad7-87f2-a6a2b3e5c4d5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:31.314586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:31.423958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:31.441278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:31.481240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed8908-e7f9-41b1-8a3e-6623ff5e90cb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:31.505081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48607f57-28ba-4ad7-87f2-a6a2b3e5c4d5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:31.525525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866bd93-4bc8-455c-875b-d210524e5eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:31.526966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866bd93-4bc8-455c-875b-d210524e5eed None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.527629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866bd93-4bc8-455c-875b-d210524e5eed None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.528180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866bd93-4bc8-455c-875b-d210524e5eed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.542560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866bd93-4bc8-455c-875b-d210524e5eed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:31.544945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866bd93-4bc8-455c-875b-d210524e5eed None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:31.569313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aecf69-1b42-4ee9-9035-c54b700180af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:31.570742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aecf69-1b42-4ee9-9035-c54b700180af None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.571474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aecf69-1b42-4ee9-9035-c54b700180af None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.572162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aecf69-1b42-4ee9-9035-c54b700180af None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9866bd93-4bc8-455c-875b-d210524e5eed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:31.572581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:31.574373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:31.576167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:31.576167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:31.576167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:31.576167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:31.577890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aecf69-1b42-4ee9-9035-c54b700180af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:31.578620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aecf69-1b42-4ee9-9035-c54b700180af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:31.579623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 233/461] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:31.581585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c968e11-61f2-4049-832f-70b6fdc5d529 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.581856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aecf69-1b42-4ee9-9035-c54b700180af None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:31.582026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c968e11-61f2-4049-832f-70b6fdc5d529 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.582401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c968e11-61f2-4049-832f-70b6fdc5d529 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.584577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 234/462] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:31.586687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c968e11-61f2-4049-832f-70b6fdc5d529 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:31.618332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 229/463] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:31.620959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eada8-d4f4-44fa-825f-b5ceaacb5d94 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.621776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eada8-d4f4-44fa-825f-b5ceaacb5d94 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.622624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eada8-d4f4-44fa-825f-b5ceaacb5d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.650740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-744eada8-d4f4-44fa-825f-b5ceaacb5d94 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:31.653311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-744eada8-d4f4-44fa-825f-b5ceaacb5d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:31.721825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c968e11-61f2-4049-832f-70b6fdc5d529 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:31.736498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:31.775703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 230/464] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:31.777876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744eada8-d4f4-44fa-825f-b5ceaacb5d94 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:31.787652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:31.789664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.790405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.791101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.792609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:31.793325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:31.796682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:31.821901 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 235/465] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:31.824553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:31.881289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:31.946399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744eada8-d4f4-44fa-825f-b5ceaacb5d94 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:31.973609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa92050-ca77-43e0-a751-67217a8788c7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:31.992057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922be85-61df-4e75-80ba-cb701f689d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:31.993030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922be85-61df-4e75-80ba-cb701f689d07 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:31.993438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922be85-61df-4e75-80ba-cb701f689d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:31.993863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922be85-61df-4e75-80ba-cb701f689d07 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:31.994921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922be85-61df-4e75-80ba-cb701f689d07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:31.996571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922be85-61df-4e75-80ba-cb701f689d07 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:32.014227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:32.019174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:32.019174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:32.019174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:32.028122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:32.029586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:32.049783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:32.060783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:32.062437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:32.062437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:32.062437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:32.062437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:32.062437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 236/466] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:32.062437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:32.081117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:32.108686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 231/467] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:43:31 2022] POST /identity/v3/users => generated 246 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:43:32.111145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eac11-c433-44d3-8545-282312ed4a7f None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:32.111794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eac11-c433-44d3-8545-282312ed4a7f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:32.112353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eac11-c433-44d3-8545-282312ed4a7f None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:32.118694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 232/468] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:32.118694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49eac11-c433-44d3-8545-282312ed4a7f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:32.246022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee822a4a-ca16-4614-b874-f0930232bc64 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:32.316220 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbdd14-25f1-428e-b6a6-2d4584783010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:32.318394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbdd14-25f1-428e-b6a6-2d4584783010 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:32.319183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbdd14-25f1-428e-b6a6-2d4584783010 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:32.319980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbdd14-25f1-428e-b6a6-2d4584783010 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:32.321713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbdd14-25f1-428e-b6a6-2d4584783010 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:32.324991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbdd14-25f1-428e-b6a6-2d4584783010 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:32.356792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49eac11-c433-44d3-8545-282312ed4a7f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:32.376585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:32.405961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84036e45-973a-47e7-9deb-3aee5956558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:32.408303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84036e45-973a-47e7-9deb-3aee5956558b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:32.408303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84036e45-973a-47e7-9deb-3aee5956558b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:32.408303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84036e45-973a-47e7-9deb-3aee5956558b None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:32.409484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84036e45-973a-47e7-9deb-3aee5956558b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:32.410025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84036e45-973a-47e7-9deb-3aee5956558b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:32.413161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84036e45-973a-47e7-9deb-3aee5956558b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:32.433027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 233/469] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:32.435060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e5068-0ee2-45df-82cf-ff890872363a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:32.435618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e5068-0ee2-45df-82cf-ff890872363a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:32.436093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e5068-0ee2-45df-82cf-ff890872363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:32.441712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 237/470] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:43:32 2022] POST /identity/v3/users => generated 248 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:43:32.444712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882e2d6-9ec8-489a-b6de-e00951583e29 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:32.445654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882e2d6-9ec8-489a-b6de-e00951583e29 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:32.446424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882e2d6-9ec8-489a-b6de-e00951583e29 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:32.449363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 238/471] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:32.455607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882e2d6-9ec8-489a-b6de-e00951583e29 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:32.461873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3a6e5068-0ee2-45df-82cf-ff890872363a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:32.464966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3a6e5068-0ee2-45df-82cf-ff890872363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:32.604925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 234/472] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:32.609160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cf248-8e1d-4776-9e19-efcc571a5085 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:32.609956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cf248-8e1d-4776-9e19-efcc571a5085 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:32.610683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cf248-8e1d-4776-9e19-efcc571a5085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:32.618101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:32.642917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-be9cf248-8e1d-4776-9e19-efcc571a5085 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:32.644863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-be9cf248-8e1d-4776-9e19-efcc571a5085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:32.667748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882e2d6-9ec8-489a-b6de-e00951583e29 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:32.728033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d28879-8abb-4612-bc6c-8fb0d1f04f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:32.728723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d28879-8abb-4612-bc6c-8fb0d1f04f7a None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:32.729015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d28879-8abb-4612-bc6c-8fb0d1f04f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:32.729290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d28879-8abb-4612-bc6c-8fb0d1f04f7a None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:32.730015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d28879-8abb-4612-bc6c-8fb0d1f04f7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:32.731512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d28879-8abb-4612-bc6c-8fb0d1f04f7a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:32.782993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 235/473] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:32.783813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cf248-8e1d-4776-9e19-efcc571a5085 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:32.832130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:32.849082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 239/474] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:43:32 2022] POST /identity/v3/users => generated 248 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:43:32.850456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d28879-8abb-4612-bc6c-8fb0d1f04f7a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:32.986021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cf248-8e1d-4776-9e19-efcc571a5085 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:33.065394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:33.065973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:33.066261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:33.066539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:33.093940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:33.094607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:33.128121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:33.129867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:33.132895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:33.132895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:33.132895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:33.132895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:33.135403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 236/475] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:33.143915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:33.161429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d28879-8abb-4612-bc6c-8fb0d1f04f7a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:33.194617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:33.201147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:33.201820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:33.202108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:33.202389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:33.203161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:33.204613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:33.243262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.sql.core [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 03:43:33.243262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 01 03:43:33.243262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [parameters: {'id': '16986b859a3843e98778b69da445f90a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 01 03:43:33.243262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 01 03:43:33.266644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return ref.to_dict() Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 03:43:33.268315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application yield resource Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise self.value Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application session.commit() Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 03:43:33.270548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.flush() Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application compat.raise_( Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 03:43:33.272336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 03:43:33.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 03:43:33.277427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [parameters: {'id': '16986b859a3843e98778b69da445f90a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:33.282631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 01 03:43:33.286916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:33.290104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 240/476] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 1 03:43:32 2022] POST /identity/v3/roles => generated 144 bytes in 419 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:33.290104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:33.343335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:33.370334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498eafc-21aa-4bd1-9dda-ee9a789ff9ba None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:33.405395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c9af6-ba1e-4c1c-a9d7-36b56992abe2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:33.436483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:33.437829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:33.438505 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:33.439156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:33.440587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:33.441281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:33.445074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:33.446981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845c699-deb7-4dd3-8f18-72aaa0a323ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:33.447906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845c699-deb7-4dd3-8f18-72aaa0a323ae None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:33.448257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845c699-deb7-4dd3-8f18-72aaa0a323ae None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:33.448622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845c699-deb7-4dd3-8f18-72aaa0a323ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:33.461212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845c699-deb7-4dd3-8f18-72aaa0a323ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:33.468660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 237/477] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d845c699-deb7-4dd3-8f18-72aaa0a323ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:33.471732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:33.473646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:33.475289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 241/478] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:43:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:33.479990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:33.482241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845c699-deb7-4dd3-8f18-72aaa0a323ae None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:33.663845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:33.750426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845c699-deb7-4dd3-8f18-72aaa0a323ae None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:33.768899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc7ffe-d249-4329-ab37-3016a9c93fdb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:33.839272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d6e9f-2f8c-46b9-a22c-ac3edb927ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:33.840561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d6e9f-2f8c-46b9-a22c-ac3edb927ea5 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:33.842080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d6e9f-2f8c-46b9-a22c-ac3edb927ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:33.843053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d6e9f-2f8c-46b9-a22c-ac3edb927ea5 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:33.844485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d6e9f-2f8c-46b9-a22c-ac3edb927ea5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:33.847546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d6e9f-2f8c-46b9-a22c-ac3edb927ea5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:33.848430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f27914-6f04-4ee6-afd6-4e5ee7002b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:33.850205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27914-6f04-4ee6-afd6-4e5ee7002b8a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:33.852088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27914-6f04-4ee6-afd6-4e5ee7002b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:33.852584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27914-6f04-4ee6-afd6-4e5ee7002b8a None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:33.853609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f27914-6f04-4ee6-afd6-4e5ee7002b8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:33.855697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f27914-6f04-4ee6-afd6-4e5ee7002b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:33.859013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f27914-6f04-4ee6-afd6-4e5ee7002b8a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:33.874477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 238/479] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:33.960428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 242/480] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:43:33 2022] POST /identity/v3/users => generated 249 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:43:36.144712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd05e-173b-4de4-9b10-1c3047b73946 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:36.145268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd05e-173b-4de4-9b10-1c3047b73946 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:36.145649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd05e-173b-4de4-9b10-1c3047b73946 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:36.149467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 239/481] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:43:36.151293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431bde1e-b629-401c-9eea-fee93913dc59 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:36.151811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431bde1e-b629-401c-9eea-fee93913dc59 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:36.152308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431bde1e-b629-401c-9eea-fee93913dc59 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:36.155642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 243/482] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:36.155979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09259ba5-93b7-4814-b83d-796bc48e1bb1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:36.156473 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09259ba5-93b7-4814-b83d-796bc48e1bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:36.156967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09259ba5-93b7-4814-b83d-796bc48e1bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:36.164440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f3dd-3885-451b-abb9-cd35640ce371 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:36.164934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f3dd-3885-451b-abb9-cd35640ce371 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:36.165418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f3dd-3885-451b-abb9-cd35640ce371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:36.200100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-09259ba5-93b7-4814-b83d-796bc48e1bb1 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:36.202186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-09259ba5-93b7-4814-b83d-796bc48e1bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:36.207606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:36.220896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-aba7f3dd-3885-451b-abb9-cd35640ce371 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:36.222943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aba7f3dd-3885-451b-abb9-cd35640ce371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:36.354304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 244/483] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:36.384738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 240/484] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:43:36.407726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7f3dd-3885-451b-abb9-cd35640ce371 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:36.492497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09259ba5-93b7-4814-b83d-796bc48e1bb1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:36.509909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:36.657271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7f3dd-3885-451b-abb9-cd35640ce371 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:36.728095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09259ba5-93b7-4814-b83d-796bc48e1bb1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:36.735628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7bbe3-6ee5-4d18-a50b-d7afdbd14ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:36.736911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7bbe3-6ee5-4d18-a50b-d7afdbd14ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:36.737354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7bbe3-6ee5-4d18-a50b-d7afdbd14ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:36.737794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7bbe3-6ee5-4d18-a50b-d7afdbd14ac6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:36.773412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7bbe3-6ee5-4d18-a50b-d7afdbd14ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3ba7bbe3-6ee5-4d18-a50b-d7afdbd14ac6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:36.794062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:36.795801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:36.797361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 245/485] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:43:36.808604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7bbe3-6ee5-4d18-a50b-d7afdbd14ac6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:36.818557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367d557-376b-4ceb-831a-1e51ff3af72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:36.820434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d557-376b-4ceb-831a-1e51ff3af72c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:36.821166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d557-376b-4ceb-831a-1e51ff3af72c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:36.822036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d557-376b-4ceb-831a-1e51ff3af72c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:36.848101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:36.859052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367d557-376b-4ceb-831a-1e51ff3af72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a367d557-376b-4ceb-831a-1e51ff3af72c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:36.883466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:36.884408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:36.885175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:36.886359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 241/486] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:36.893244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367d557-376b-4ceb-831a-1e51ff3af72c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:36.990196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7bbe3-6ee5-4d18-a50b-d7afdbd14ac6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:36.998309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:37.021637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367d557-376b-4ceb-831a-1e51ff3af72c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:37.062839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8499b09-64d6-4bfe-ac2a-8bca56433ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:37.063892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8499b09-64d6-4bfe-ac2a-8bca56433ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.064492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8499b09-64d6-4bfe-ac2a-8bca56433ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.064943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8499b09-64d6-4bfe-ac2a-8bca56433ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.065717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8499b09-64d6-4bfe-ac2a-8bca56433ab5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:37.066027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8499b09-64d6-4bfe-ac2a-8bca56433ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:37.067439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8499b09-64d6-4bfe-ac2a-8bca56433ab5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:37.079338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 242/487] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:37.083392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31377d49-d6ee-4c70-a2e2-24939c283cc7 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.084115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31377d49-d6ee-4c70-a2e2-24939c283cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.084814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31377d49-d6ee-4c70-a2e2-24939c283cc7 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.088055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 243/488] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:37.089865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31377d49-d6ee-4c70-a2e2-24939c283cc7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:37.092608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fff05-238c-4fba-a9ca-c30713ff51c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:37.093375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fff05-238c-4fba-a9ca-c30713ff51c8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.093785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fff05-238c-4fba-a9ca-c30713ff51c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.094223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fff05-238c-4fba-a9ca-c30713ff51c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.095081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1fff05-238c-4fba-a9ca-c30713ff51c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:37.095794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1fff05-238c-4fba-a9ca-c30713ff51c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:37.097080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1fff05-238c-4fba-a9ca-c30713ff51c8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:37.107096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 246/489] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:37.109564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ed836-85af-43c7-a3d2-01b4bb0a1ee9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.109564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ed836-85af-43c7-a3d2-01b4bb0a1ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.109564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ed836-85af-43c7-a3d2-01b4bb0a1ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.130674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2c4ed836-85af-43c7-a3d2-01b4bb0a1ee9 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:37.132144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2c4ed836-85af-43c7-a3d2-01b4bb0a1ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:37.211530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:37.212454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 247/490] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:37.214617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ed836-85af-43c7-a3d2-01b4bb0a1ee9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:37.296101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31377d49-d6ee-4c70-a2e2-24939c283cc7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:37.330084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:37.340485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ed836-85af-43c7-a3d2-01b4bb0a1ee9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:37.360735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:37.362289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.362802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.363328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.381780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c997f9-ef2c-4326-a73b-11815da65442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:37.382867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c997f9-ef2c-4326-a73b-11815da65442 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.383371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c997f9-ef2c-4326-a73b-11815da65442 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.383719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c997f9-ef2c-4326-a73b-11815da65442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.389001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:37.389001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:37.396092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c997f9-ef2c-4326-a73b-11815da65442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:37.397562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c997f9-ef2c-4326-a73b-11815da65442 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:37.418578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:37.420563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:37.422329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:37.422329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:37.422329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:37.422329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:37.424710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 244/491] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:37.425556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-87c997f9-ef2c-4326-a73b-11815da65442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:37.428135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:37.431466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:37.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:37.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:37.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:37.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:37.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 248/492] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:37.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01364a-1a6c-4252-a8d4-88de79aeeae8 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01364a-1a6c-4252-a8d4-88de79aeeae8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01364a-1a6c-4252-a8d4-88de79aeeae8 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.439563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 249/493] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:37.439563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01364a-1a6c-4252-a8d4-88de79aeeae8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:37.583005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9e31-39fc-41e7-97e0-de7986f6923c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:37.595776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01364a-1a6c-4252-a8d4-88de79aeeae8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:37.609088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:37.639869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:37.640532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.640813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.641091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.641831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:37.642146 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:37.643618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:37.648104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91cae2-be45-42f4-893f-f51872f68753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:37.649465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91cae2-be45-42f4-893f-f51872f68753 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.649966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91cae2-be45-42f4-893f-f51872f68753 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.650462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91cae2-be45-42f4-893f-f51872f68753 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.660727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 250/494] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:37.660727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:37.666164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:37.670733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91cae2-be45-42f4-893f-f51872f68753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7d91cae2-be45-42f4-893f-f51872f68753 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:37.687278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:37.688238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:37.689113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:37.689717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 245/495] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:37.693856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51409f12-6532-47a6-a50c-562065a12ef7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.693856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51409f12-6532-47a6-a50c-562065a12ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.693856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51409f12-6532-47a6-a50c-562065a12ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.716332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-51409f12-6532-47a6-a50c-562065a12ef7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:37.718066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-51409f12-6532-47a6-a50c-562065a12ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:37.801575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 246/496] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:37.802563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51409f12-6532-47a6-a50c-562065a12ef7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:37.820547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a28f2-04e8-4e91-be29-f4c0ff184eba None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:37.875009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:37.875612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.876004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.876154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.876904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:37.877280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:37.878648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:37.894694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 251/497] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:37.894694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:37.926459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51409f12-6532-47a6-a50c-562065a12ef7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:37.928994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:37.962488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:37.964458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:37.965262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:37.965545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:37.965820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:37.973826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:37.975508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:37.993535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:37.995587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 247/498] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:43:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:37.997759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.010064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ceae9-0c90-4e64-8ff5-f9bde0c4f4d9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.049390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.050082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.050356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.050630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.051432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:38.051763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.053671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.062849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 252/499] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:38.063751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.121193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03f1e7-94dc-421b-8420-98d716d65fe8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.158206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78353f5-1664-46dc-b2eb-6e8d5ba781c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.159007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78353f5-1664-46dc-b2eb-6e8d5ba781c7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.159383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78353f5-1664-46dc-b2eb-6e8d5ba781c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.159725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78353f5-1664-46dc-b2eb-6e8d5ba781c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.171891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78353f5-1664-46dc-b2eb-6e8d5ba781c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.173162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492487a-171a-4e3a-a7be-9796176d65cf None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.173379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78353f5-1664-46dc-b2eb-6e8d5ba781c7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a78353f5-1664-46dc-b2eb-6e8d5ba781c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:38.192012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:38.193734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:38.195637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:38.195637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:38.195637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:38.195637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:38.195637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 248/500] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:38.195637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110aa69-6ad3-4235-9d90-b1c97807634c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.195637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110aa69-6ad3-4235-9d90-b1c97807634c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.195637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110aa69-6ad3-4235-9d90-b1c97807634c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.196659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 249/501] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:38.197399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110aa69-6ad3-4235-9d90-b1c97807634c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.209753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:38.214835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:38.218730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.219570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.224936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.224936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.228897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.230357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:38.248409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:38.249883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 03:43:38.250938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:38.254164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 253/502] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:38.254164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.315490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110aa69-6ad3-4235-9d90-b1c97807634c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.353156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.353826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.354103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.354386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.355264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:38.355564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.357402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.362708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3d2fa-57a8-4da9-a760-2dd9dcfb0430 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.396958 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 250/503] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:43:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:38.397816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.405201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7b07-9f19-4532-9eec-1aa080feeb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.406006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7b07-9f19-4532-9eec-1aa080feeb98 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.406289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7b07-9f19-4532-9eec-1aa080feeb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.406565 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7b07-9f19-4532-9eec-1aa080feeb98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.418718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed7b07-9f19-4532-9eec-1aa080feeb98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.420146 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed7b07-9f19-4532-9eec-1aa080feeb98 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-04ed7b07-9f19-4532-9eec-1aa080feeb98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:38.438958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:38.439974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:38.441334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:38.441334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:38.441334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:38.441334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:38.441334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 254/504] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:38.441861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c30c50-a6cc-47a4-bc39-8225bd23ba8c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.442759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c30c50-a6cc-47a4-bc39-8225bd23ba8c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.442759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c30c50-a6cc-47a4-bc39-8225bd23ba8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.459574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-83c30c50-a6cc-47a4-bc39-8225bd23ba8c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:38.460819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-83c30c50-a6cc-47a4-bc39-8225bd23ba8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:38.483330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:38.498551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:38.516958 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06437bb-14ad-45f1-a869-5580d61a3c46 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.541387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 255/505] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:38.542258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c30c50-a6cc-47a4-bc39-8225bd23ba8c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.556513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.557311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.557594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.557870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.558616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:38.558925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.560385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.573410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 251/506] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:38.574579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.674520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c30c50-a6cc-47a4-bc39-8225bd23ba8c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.689533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4b983-4502-4f5e-84fb-4f362c89fd89 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.717076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.718034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.718431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.718858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.719844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:38.720327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.722190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.724795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:38.729981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1323a5-15ec-4902-8460-4b70c86ece53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.730641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1323a5-15ec-4902-8460-4b70c86ece53 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.730924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1323a5-15ec-4902-8460-4b70c86ece53 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.731367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1323a5-15ec-4902-8460-4b70c86ece53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.763464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 256/507] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 1 03:43:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:38.764329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.797315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:38.806145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1323a5-15ec-4902-8460-4b70c86ece53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.807702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1323a5-15ec-4902-8460-4b70c86ece53 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.880455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfd242-3449-4971-8175-e522dcc2a16b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:38.925864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 252/508] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:38.927164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1323a5-15ec-4902-8460-4b70c86ece53 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.928304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:38.929566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:38.930848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:38.931491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:38.932640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:38.933000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:38.935363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:38.953241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 257/509] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:38.954092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:38.968580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:39.043735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1323a5-15ec-4902-8460-4b70c86ece53 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.048169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:39.064501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507729-1cf0-4b16-b111-ca89542ea2fc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.085034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf000f-35d1-45d4-b9d1-c632968182bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:39.085843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf000f-35d1-45d4-b9d1-c632968182bb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:39.086126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf000f-35d1-45d4-b9d1-c632968182bb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:39.086402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf000f-35d1-45d4-b9d1-c632968182bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:39.097745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf000f-35d1-45d4-b9d1-c632968182bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:39.102750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771c538-9389-4f75-bb83-d673407084b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:39.102750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c538-9389-4f75-bb83-d673407084b2 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:39.103746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c538-9389-4f75-bb83-d673407084b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:39.103746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c538-9389-4f75-bb83-d673407084b2 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:39.105390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771c538-9389-4f75-bb83-d673407084b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:39.105390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771c538-9389-4f75-bb83-d673407084b2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c9bf000f-35d1-45d4-b9d1-c632968182bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:39.105660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:39.114603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 253/510] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:39.116445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf000f-35d1-45d4-b9d1-c632968182bb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:39.185232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 258/511] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:43:38 2022] POST /identity/v3/users => generated 247 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:43:39.185995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771c538-9389-4f75-bb83-d673407084b2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:39.236362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:39.250523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf000f-35d1-45d4-b9d1-c632968182bb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.304342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5bc7-fd58-49ec-acf2-746614adb63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:39.305795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5bc7-fd58-49ec-acf2-746614adb63d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:39.306074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5bc7-fd58-49ec-acf2-746614adb63d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:39.306457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5bc7-fd58-49ec-acf2-746614adb63d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:39.329492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771c538-9389-4f75-bb83-d673407084b2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.377257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:39.378055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:39.378336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:39.378607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:39.392545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:39.394001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:39.396581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:39.405259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c5bc7-fd58-49ec-acf2-746614adb63d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:39.406740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c5bc7-fd58-49ec-acf2-746614adb63d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:39.414292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:39.415157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:39.416135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:39.416135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:39.416135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:39.416135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:39.416135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 259/512] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:39.416388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:39.519052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:39.530097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 254/513] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:39.531347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5bc7-fd58-49ec-acf2-746614adb63d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:39.543456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b163a2-e435-48d0-ae49-8cb451fe6d8f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.590024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:39.590818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:39.591102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:39.591425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:39.601053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:39.602529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:39.620558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:39.621990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 260/514] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:39.622918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:39.678923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5bc7-fd58-49ec-acf2-746614adb63d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.695708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:39.734317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:39.735098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:39.735421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:39.735903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:39.736564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:39.736761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:39.738142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:39.750066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 255/515] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:39.752258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:39.765264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819630e-6630-40a5-a63b-cf96000abc1a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.806746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4324555-c22f-4211-a515-6552940b933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:39.807854 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4324555-c22f-4211-a515-6552940b933a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:39.808363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4324555-c22f-4211-a515-6552940b933a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:39.808868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4324555-c22f-4211-a515-6552940b933a None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:39.810267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4324555-c22f-4211-a515-6552940b933a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:39.810829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4324555-c22f-4211-a515-6552940b933a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:39.813587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4324555-c22f-4211-a515-6552940b933a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:39.829977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:39.837763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 261/516] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:39.839364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4324555-c22f-4211-a515-6552940b933a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:39.903211 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6decdd-feae-429d-be92-d6d4f918de5b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.943428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:39.944586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:39.945067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:39.945431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:39.956960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4324555-c22f-4211-a515-6552940b933a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:39.959288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:39.960924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:39.974534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:39.982517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:39.992277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:39.993353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:39.993353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:39.993353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:39.993353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:39.993353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 256/517] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:39.993353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:40.012307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:40.013066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:40.013373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:40.013706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:40.015193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:40.015787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:40.017771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:40.055881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 262/518] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 1 03:43:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:40.057080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:40.159570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c39a2d-204f-4456-989b-9ca2f9dd42a8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:40.167255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:40.197352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:40.198139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:40.198422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:40.198691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:40.200780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d92a-c9c6-4bc8-80ba-a184f20ab15b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:40.209777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:40.211288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:40.230112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:40.232229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:40.234084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:40.234084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:40.234084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:40.234084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:40.234084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 257/519] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:40.234084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:40.248216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:40.249587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:40.250075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:40.250567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:40.260657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:40.263055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:40.289852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:40.290878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:40.291950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:40.291950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:40.291950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:40.291950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:40.291950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:40.291950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:40.291950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:40.291950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:40.292677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 263/520] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:40.296254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:40.337551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:40.373147 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bbb6c-0a31-4c7d-a2f8-733ac5d4daec None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:40.418349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:40.419033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:40.419348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:40.419631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:40.420379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:40.420689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:40.421777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d016c-fc48-453f-8a59-c7e4a6b2470a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:40.422093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:40.440976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 258/521] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:40.441844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:40.464642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:40.479348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9a88d-ff03-4663-bc5a-9d15c9093b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:40.480073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9a88d-ff03-4663-bc5a-9d15c9093b32 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:40.480360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9a88d-ff03-4663-bc5a-9d15c9093b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:40.480637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9a88d-ff03-4663-bc5a-9d15c9093b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:40.560288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9a88d-ff03-4663-bc5a-9d15c9093b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:40.561777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9a88d-ff03-4663-bc5a-9d15c9093b32 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:40.566296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e7ff-0f1e-44a7-98e9-e9f618090ddb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:40.594521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:40.626765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:40.628237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:40.628723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:40.629037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:40.629966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:40.630515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:40.633163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:40.660073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 259/522] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:40.660073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:40.683538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 264/523] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:40.684973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9a88d-ff03-4663-bc5a-9d15c9093b32 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:40.777419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:40.808102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66f075-c786-4ca7-9ceb-b61e2c821386 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:40.834665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9a88d-ff03-4663-bc5a-9d15c9093b32 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:40.848563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:40.849147 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:40.849438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:40.849718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:40.850527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:40.850838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:40.852366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:40.877497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 260/524] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:40.877497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:40.894061 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:40.904567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:40.906176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:40.906676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:40.907179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:40.919997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:40.922236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:40.940105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:40.941005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:40.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 265/525] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:40.943874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:41.015704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21475a6-e0e5-49aa-b82c-081c290d76d6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:41.053819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:41.054611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.055145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.055444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.063816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:41.065281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:41.067571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:41.081160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:41.088222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 261/526] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:41.089412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:41.125910 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4005e85-8fcf-43be-8110-650716f43b9b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:41.169240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:41.170509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.171002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.171507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.184196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:41.185961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:41.206607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:41.208826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:41.209735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:41.209735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:41.209735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:41.209735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:41.209953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f71765-391d-46da-9f18-fd792cccd522 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:41.213147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 266/527] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:41.214137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:41.257641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:41.258580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.258860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.259146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.260039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=dbd31b58571b476bad12735ff7445d55) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:41.260527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:41.262237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:41.269185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 262/528] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:41 2022] GET /identity/v3/role_assignments?user.id=dbd31b58571b476bad12735ff7445d55&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:41.271266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:41.280550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:41.377153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:41.392577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadf72a-8391-41ab-99a9-f9420b2460ef None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:41.460678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:41.461861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.462209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.462635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.464217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:41.464789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:41.465637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e32ab4-337e-4cc2-b3ec-1c5b13f4de52 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:41.467404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:41.502233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 267/529] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:41.503272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:41.538276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6e408-fe6f-4c40-96f2-52309bc42caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:41.539461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e408-fe6f-4c40-96f2-52309bc42caf None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.539955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e408-fe6f-4c40-96f2-52309bc42caf None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.540450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e408-fe6f-4c40-96f2-52309bc42caf None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.542060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6e408-fe6f-4c40-96f2-52309bc42caf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:41.542614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6e408-fe6f-4c40-96f2-52309bc42caf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:41.545372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6e408-fe6f-4c40-96f2-52309bc42caf None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:41.576633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 263/530] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Jul 1 03:43:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:41.579445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe702cf3-b1a1-4737-83b3-9c2c0e1a8452 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.579580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe702cf3-b1a1-4737-83b3-9c2c0e1a8452 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.580273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe702cf3-b1a1-4737-83b3-9c2c0e1a8452 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.582846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 264/531] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:41.584489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe702cf3-b1a1-4737-83b3-9c2c0e1a8452 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:41.648932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b88c7-431f-496e-8f0c-22e6b547d7c1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:41.652941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:41.703342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:41.706665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.706665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.706665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.706982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:41.707314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:41.709753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:41.724566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 268/532] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:41.725171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:41.760553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe702cf3-b1a1-4737-83b3-9c2c0e1a8452 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:41.786271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:41.802193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352b79b-9bd8-49d3-9aa2-ea05d80c66eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:41.802979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b79b-9bd8-49d3-9aa2-ea05d80c66eb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.803361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b79b-9bd8-49d3-9aa2-ea05d80c66eb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.803703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b79b-9bd8-49d3-9aa2-ea05d80c66eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.804626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352b79b-9bd8-49d3-9aa2-ea05d80c66eb None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=e8327d9916424843b2a67cbdd856ad37) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:41.805262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352b79b-9bd8-49d3-9aa2-ea05d80c66eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:41.807094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352b79b-9bd8-49d3-9aa2-ea05d80c66eb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:41.819653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 265/533] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:41 2022] GET /identity/v3/role_assignments?user.id=e8327d9916424843b2a67cbdd856ad37&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:41.823164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b056caf-6cce-4aa3-8c45-5796e695a61b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.824333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b056caf-6cce-4aa3-8c45-5796e695a61b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.824788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b056caf-6cce-4aa3-8c45-5796e695a61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.847703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9b056caf-6cce-4aa3-8c45-5796e695a61b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:41.849931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9b056caf-6cce-4aa3-8c45-5796e695a61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:41.867183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cd50c-6d81-43a9-918c-f02435fd7c9c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:41.923792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a7860-4c0b-4c63-98e8-ecb10c88b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:41.925405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a7860-4c0b-4c63-98e8-ecb10c88b50f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:41.925864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a7860-4c0b-4c63-98e8-ecb10c88b50f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:41.926279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a7860-4c0b-4c63-98e8-ecb10c88b50f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:41.972358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:41.975167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 266/534] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:41.976960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b056caf-6cce-4aa3-8c45-5796e695a61b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:42.029344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a7860-4c0b-4c63-98e8-ecb10c88b50f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:42.030812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a7860-4c0b-4c63-98e8-ecb10c88b50f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:42.172144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b056caf-6cce-4aa3-8c45-5796e695a61b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:42.187507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 269/535] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:42.188512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a7860-4c0b-4c63-98e8-ecb10c88b50f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:42.198754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:42.225011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:42.226091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:42.226624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:42.226999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:42.235728 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:42.237947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:42.264625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:42.265508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:42.266411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:42.266411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:42.266411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:42.266411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:42.266411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:42.266411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:42.266411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:42.266411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:42.266991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 267/536] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:42.269241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:42.297202 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:42.364239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a7860-4c0b-4c63-98e8-ecb10c88b50f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:42.415517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9307f7-fdbb-4a38-8e0f-55281ec6d340 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:42.435153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721521cb-c5fa-47f9-b797-510916e1041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:42.436206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721521cb-c5fa-47f9-b797-510916e1041e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:42.436828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721521cb-c5fa-47f9-b797-510916e1041e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:42.437318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721521cb-c5fa-47f9-b797-510916e1041e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:42.452968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721521cb-c5fa-47f9-b797-510916e1041e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-721521cb-c5fa-47f9-b797-510916e1041e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:42.463618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:42.464567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:42.465358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 270/537] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:42.466502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721521cb-c5fa-47f9-b797-510916e1041e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:42.472121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:42.473232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:42.474170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:42.474671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:42.490376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:42.492260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:42.512498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:42.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:42.527558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:42.527558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:42.527558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:42.527558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:42.527558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 268/538] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:42.527558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:42.549490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:42.616020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721521cb-c5fa-47f9-b797-510916e1041e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:42.649230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:42.664202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58603724-290e-4ce0-83d9-b25de51da99b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:42.664752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:42.665719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:42.666157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:42.666920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:42.668986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:42.669482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:42.671189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:42.689699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 271/539] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:42.690822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:42.726785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:42.727657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:42.728041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:42.728415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:42.729289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:42.729698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:42.731193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:42.744678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 269/540] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:42.746172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:42.833925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c852cc2-435e-417a-9f6b-e163a0b26f01 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:42.879136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:42.880813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:42.881240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:42.881653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:42.883259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:42.883865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:42.885618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:42.890350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:42.902515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 272/541] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:42.903876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:42.908513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29d8d3-3079-431e-9709-4da3a922f220 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:42.967917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:42.973105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:42.974363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:42.974780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:42.975157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:42.976150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=ec8a755e84f04c9590c9c8d57781d2a8) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:42.976566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:42.978140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:42.993420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 270/542] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:42 2022] GET /identity/v3/role_assignments?user.id=ec8a755e84f04c9590c9c8d57781d2a8&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:42.995241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:43.049432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ee00c-89ae-4044-a303-b1e816cc5889 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:43.094733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:43.095972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:43.096698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:43.097251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:43.113849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:43.116327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:43.141897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:43.142897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:43.144197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:43.144197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:43.144197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:43.144197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:43.147751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 273/543] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:43.154499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:43.184219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9041d-e0f2-44c2-9ee1-a461f1f0149c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:43.287051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:43.291467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:43.291803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:43.292169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:43.309594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:43.311483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:43.344503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:43.346166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:43.347875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 271/544] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:43.353213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:43.381027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:43.418756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:43.435732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07efe4e-9cb7-4e27-92bc-3da8636273b8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:43.523130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:43.524712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:43.525451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:43.526221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:43.528413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:43.529354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:43.532044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:43.555317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 274/545] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:43.561197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:43.627993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2439a6b-7814-4234-9772-2a4197ce20df None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:43.681002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff69371-bf40-4445-b17e-4ad3e5c86e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:43.681679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff69371-bf40-4445-b17e-4ad3e5c86e96 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:43.681964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff69371-bf40-4445-b17e-4ad3e5c86e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:43.682239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff69371-bf40-4445-b17e-4ad3e5c86e96 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:43.683055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff69371-bf40-4445-b17e-4ad3e5c86e96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:43.683418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff69371-bf40-4445-b17e-4ad3e5c86e96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:43.685614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff69371-bf40-4445-b17e-4ad3e5c86e96 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:43.714149 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 272/546] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:43.715862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2cec3-a3a3-4a4c-922f-9c0b012de228 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:43.716400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2cec3-a3a3-4a4c-922f-9c0b012de228 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:43.716834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2cec3-a3a3-4a4c-922f-9c0b012de228 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:43.718677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 273/547] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:43.719969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2cec3-a3a3-4a4c-922f-9c0b012de228 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:43.731307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5a145-7c87-4abb-9267-40538f35819a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:43.772959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972846-0bca-4038-b902-e829683e18a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:43.773213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972846-0bca-4038-b902-e829683e18a2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:43.773448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972846-0bca-4038-b902-e829683e18a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:43.773792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972846-0bca-4038-b902-e829683e18a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:43.782385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972846-0bca-4038-b902-e829683e18a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:43.784011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972846-0bca-4038-b902-e829683e18a2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1f972846-0bca-4038-b902-e829683e18a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:43.802396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:43.803620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:43:43.804961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:43.810178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 275/548] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:43:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:43.811749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719c11c-af94-4be0-869a-80b118b18d66 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:43.812107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719c11c-af94-4be0-869a-80b118b18d66 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:43.812410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719c11c-af94-4be0-869a-80b118b18d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:43.826472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:43.832846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9719c11c-af94-4be0-869a-80b118b18d66 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:43.835015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9719c11c-af94-4be0-869a-80b118b18d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:43.852131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:43.861625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2cec3-a3a3-4a4c-922f-9c0b012de228 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:43.904964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f1205-a262-418b-8939-63caee85e326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:43.905777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f1205-a262-418b-8939-63caee85e326 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:43.906174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f1205-a262-418b-8939-63caee85e326 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:43.906522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f1205-a262-418b-8939-63caee85e326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:43.907484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f1205-a262-418b-8939-63caee85e326 None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=0beff646206e446386d91537df9eaa67) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:43.907870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f1205-a262-418b-8939-63caee85e326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:43.909416 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f1205-a262-418b-8939-63caee85e326 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:43.919793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 274/549] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:43 2022] GET /identity/v3/role_assignments?user.id=0beff646206e446386d91537df9eaa67&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:43.921624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f1205-a262-418b-8939-63caee85e326 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:43.933262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 276/550] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:43.979025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719c11c-af94-4be0-869a-80b118b18d66 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:44.136088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f1205-a262-418b-8939-63caee85e326 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:44.195392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719c11c-af94-4be0-869a-80b118b18d66 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:44.196532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69e6ac-a8ad-4308-950a-d177d2681a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:44.197235 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e6ac-a8ad-4308-950a-d177d2681a30 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:44.197523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e6ac-a8ad-4308-950a-d177d2681a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:44.197804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e6ac-a8ad-4308-950a-d177d2681a30 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:44.198673 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69e6ac-a8ad-4308-950a-d177d2681a30 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:44.198993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69e6ac-a8ad-4308-950a-d177d2681a30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:44.200515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69e6ac-a8ad-4308-950a-d177d2681a30 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:44.205312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:44.226960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 275/551] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 1 03:43:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:44.228481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6706ab2-d056-457d-be15-616e0587bd3a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:44.228771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6706ab2-d056-457d-be15-616e0587bd3a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:44.229046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6706ab2-d056-457d-be15-616e0587bd3a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:44.234611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 276/552] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:44.234611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6706ab2-d056-457d-be15-616e0587bd3a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:44.249163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fdf0c-0fef-4d1c-a764-a47a4e789935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:44.251150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdf0c-0fef-4d1c-a764-a47a4e789935 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:44.251712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdf0c-0fef-4d1c-a764-a47a4e789935 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:44.252223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdf0c-0fef-4d1c-a764-a47a4e789935 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:44.272948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fdf0c-0fef-4d1c-a764-a47a4e789935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-bd8fdf0c-0fef-4d1c-a764-a47a4e789935 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:44.282765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:44.283828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:44.284619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:44.285234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 277/553] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:43:44.287128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b2648-551b-4145-9729-d96dd8c1bb3e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:44.287466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b2648-551b-4145-9729-d96dd8c1bb3e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:44.287743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b2648-551b-4145-9729-d96dd8c1bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:44.307547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:44.313226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2d9b2648-551b-4145-9729-d96dd8c1bb3e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:44.314557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2d9b2648-551b-4145-9729-d96dd8c1bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:44.375197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6706ab2-d056-457d-be15-616e0587bd3a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:44.419377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 278/554] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:44.419377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b2648-551b-4145-9729-d96dd8c1bb3e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:44.432941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422975d-d5c6-4b73-9eb9-2dee8fc23fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:44.433606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422975d-d5c6-4b73-9eb9-2dee8fc23fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:44.433891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422975d-d5c6-4b73-9eb9-2dee8fc23fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:44.434164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422975d-d5c6-4b73-9eb9-2dee8fc23fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:44.508128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:44.516579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422975d-d5c6-4b73-9eb9-2dee8fc23fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:44.518418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422975d-d5c6-4b73-9eb9-2dee8fc23fc2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:44.574983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b2648-551b-4145-9729-d96dd8c1bb3e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:44.637740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:44.638895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:44.639317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:44.640123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:44.641217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:44.641772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:44.644663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:44.652393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 277/555] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:44.654005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422975d-d5c6-4b73-9eb9-2dee8fc23fc2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:44.655520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:44.661695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 279/556] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:44.662602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:44.781315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cdb68-e839-4c32-81c9-9dba288f7dee None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:44.793749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422975d-d5c6-4b73-9eb9-2dee8fc23fc2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:44.826786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:44.828314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:44.828841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:44.829299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:44.833279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102931e-8e62-47c2-bc07-3ae0d9052110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:44.834096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102931e-8e62-47c2-bc07-3ae0d9052110 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:44.834378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102931e-8e62-47c2-bc07-3ae0d9052110 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:44.834647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102931e-8e62-47c2-bc07-3ae0d9052110 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:44.844290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:44.846154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:44.846435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102931e-8e62-47c2-bc07-3ae0d9052110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-3102931e-8e62-47c2-bc07-3ae0d9052110 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:44.855018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:44.868878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 278/557] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:44.872299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102931e-8e62-47c2-bc07-3ae0d9052110 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:44.878922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:44.879088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:44.890870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:44.892102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:44.892102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:44.892102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:44.892102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:44.892102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 280/558] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:44.892102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:44.957609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:45.020962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140776-7035-4d7d-8b4f-ceeb9a520146 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.027478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102931e-8e62-47c2-bc07-3ae0d9052110 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.062037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.063293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.063870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.064402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.065522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:45.066045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.068350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.077363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.078220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.078504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.078776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.079705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 281/559] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:45.080872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.096281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.098109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:45.117469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:45.118492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:45.119417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:45.119417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:45.119417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:45.119417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:45.126319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 279/560] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:45.127619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.138407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:45.199078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03438f-484d-4ccf-ba1b-e12aad9c6b66 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.213516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:45.235793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.236722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.237151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.237597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.238480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:45.239466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.241046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.255687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 282/561] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:45.256804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.259510 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aebfbc-3442-4af9-bcb6-9d03f165a71e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.298523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.299430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.299830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.300206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.314646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.316423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:45.336853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:45.337896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:45.338745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:45.338745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:45.338745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:45.338745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:45.343054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 280/562] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:45.344032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.370700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa002884-2c8a-417b-9069-0ea5dc06542f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.408183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.409242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.409740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.410214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.411191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:45.411847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.412854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:45.413792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.427575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 283/563] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:45.428754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.462905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:45.470270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7527740-fcd3-4b37-af34-8dec564db9dc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.515835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.516662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.516969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.517267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.526686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.528156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:45.545528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:45.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:45.547393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 281/564] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:45.548378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.553415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee99bf-d69a-47f5-a746-c2a6fedb59cb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.594608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.596132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.596644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.597140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.598098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:45.598713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.600429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.619128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 284/565] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:45.620498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.715507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892999c0-87e5-4062-9099-3a10ede8647b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.729628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:45.740674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c635e-1cb9-47df-8724-1346426a10c6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.748391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:45.764383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.765183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.765468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.765746 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.774112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.775624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.791931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b500b-6dd5-4d0d-812a-da1bb97b0fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:45.792302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:45.793417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:43:45.794548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:45.795101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b500b-6dd5-4d0d-812a-da1bb97b0fdd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.795101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b500b-6dd5-4d0d-812a-da1bb97b0fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.795101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b500b-6dd5-4d0d-812a-da1bb97b0fdd None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.795512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b500b-6dd5-4d0d-812a-da1bb97b0fdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:45.796071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b500b-6dd5-4d0d-812a-da1bb97b0fdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:45.797844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 282/566] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:45.798914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.799035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b500b-6dd5-4d0d-812a-da1bb97b0fdd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:45.827892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 285/567] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:45.827892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e1ed-a4b6-43eb-b015-3cc70ec96291 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:45.827892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e1ed-a4b6-43eb-b015-3cc70ec96291 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:45.827892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e1ed-a4b6-43eb-b015-3cc70ec96291 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:45.829194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 286/568] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:45.830580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e1ed-a4b6-43eb-b015-3cc70ec96291 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:45.961930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e1ed-a4b6-43eb-b015-3cc70ec96291 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:45.965899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719186ac-07c9-4c48-acdd-155d5bbfcef1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:46.001601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:46.002276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.002550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.002846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.003728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:46.004041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:46.005542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:46.026193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 287/569] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:46.027203 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:46.041568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa7c18-4060-44e7-abf4-53b5ec3912c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:46.042570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:46.043502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7c18-4060-44e7-abf4-53b5ec3912c4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.044000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7c18-4060-44e7-abf4-53b5ec3912c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.044461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7c18-4060-44e7-abf4-53b5ec3912c4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.054189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa7c18-4060-44e7-abf4-53b5ec3912c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:46.056078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa7c18-4060-44e7-abf4-53b5ec3912c4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-8caa7c18-4060-44e7-abf4-53b5ec3912c4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:46.078829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:46.080021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 03:43:46.081179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:46.084360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 283/570] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:46.086533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aa1ed-b602-4116-9d10-0152cdec3b07 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.086989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aa1ed-b602-4116-9d10-0152cdec3b07 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.087781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aa1ed-b602-4116-9d10-0152cdec3b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.112935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-427aa1ed-b602-4116-9d10-0152cdec3b07 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:46.114910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-427aa1ed-b602-4116-9d10-0152cdec3b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:46.150783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a15bc1-59c4-4b7e-be81-bccc9983d37d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:46.152976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:46.195252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:46.195921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.196204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.196481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.197318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=1123437cabbd4aa09ca3b1a0996a8bfc) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:46.197635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:46.199090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:46.202360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 284/571] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:46.203516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427aa1ed-b602-4116-9d10-0152cdec3b07 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:46.211410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 288/572] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:46 2022] GET /identity/v3/role_assignments?user.id=1123437cabbd4aa09ca3b1a0996a8bfc&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:46.212551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:46.366293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:46.373461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427aa1ed-b602-4116-9d10-0152cdec3b07 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:46.417493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bfe2f-ab7c-47aa-994a-b43bd6970c00 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:46.422208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:46.424380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.424889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.425396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.438898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:46.441313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:46.480103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:46.481848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:46.483628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:46.483628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:46.483628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:46.483628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:46.484085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:46.484235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.485481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.485999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.487716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:46.488495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:46.488770 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 285/573] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:46.490544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:46.498080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:46.539525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 289/574] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 1 03:43:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:46.540429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:46.541601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:46.679355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27ac23-8d0d-4c95-af22-aeb94d774a6f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:46.721911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56d4e5-21fb-46c5-988d-c8e24fea8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:46.722896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56d4e5-21fb-46c5-988d-c8e24fea8a0f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.723016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56d4e5-21fb-46c5-988d-c8e24fea8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.723834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56d4e5-21fb-46c5-988d-c8e24fea8a0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.737382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56d4e5-21fb-46c5-988d-c8e24fea8a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-6f56d4e5-21fb-46c5-988d-c8e24fea8a0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:46.745920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:46.748214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 286/575] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:46.749851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56d4e5-21fb-46c5-988d-c8e24fea8a0f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:46.765170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711cbab-eae4-42e1-893c-ee51281babd0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:46.804494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:46.813968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e255532-5a78-4460-a375-588089138bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:46.819123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e255532-5a78-4460-a375-588089138bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.819123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e255532-5a78-4460-a375-588089138bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.819123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e255532-5a78-4460-a375-588089138bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.819123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e255532-5a78-4460-a375-588089138bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:46.819123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e255532-5a78-4460-a375-588089138bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:46.822049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e255532-5a78-4460-a375-588089138bc8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:46.844663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 290/576] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:46.847194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac308834-63e2-48df-910b-b3a3ff9e9d6c None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.847726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac308834-63e2-48df-910b-b3a3ff9e9d6c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.848215 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac308834-63e2-48df-910b-b3a3ff9e9d6c None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:46.851635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 291/577] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:46.853050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac308834-63e2-48df-910b-b3a3ff9e9d6c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:46.881598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:46.915352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56d4e5-21fb-46c5-988d-c8e24fea8a0f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:46.981892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:46.983143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:46.983651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:46.984125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:47.004964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:47.007789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:47.011752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac308834-63e2-48df-910b-b3a3ff9e9d6c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:47.030280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:47.032075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:47.033167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:47.033167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:47.033167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:47.033167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:47.033167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 287/578] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:47.033167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:47.081765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fb6af-313f-4cc4-b9c6-3fb13ef2743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:47.083654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fb6af-313f-4cc4-b9c6-3fb13ef2743b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:47.084374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fb6af-313f-4cc4-b9c6-3fb13ef2743b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:47.084946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fb6af-313f-4cc4-b9c6-3fb13ef2743b None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:47.086501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fb6af-313f-4cc4-b9c6-3fb13ef2743b None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:47.087047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fb6af-313f-4cc4-b9c6-3fb13ef2743b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:47.090129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fb6af-313f-4cc4-b9c6-3fb13ef2743b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:47.108192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 292/579] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:47.110757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332a08b-37f3-47f3-b0e6-9bb804384967 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:47.111299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332a08b-37f3-47f3-b0e6-9bb804384967 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:47.111805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332a08b-37f3-47f3-b0e6-9bb804384967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:47.144497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7332a08b-37f3-47f3-b0e6-9bb804384967 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:47.147133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7332a08b-37f3-47f3-b0e6-9bb804384967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:47.163127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a68170-846e-4789-8b0a-545b4211ebbb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:47.210216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:47.222248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:47.230886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:47.239726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 293/580] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:47.239956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:47.239956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:47.240234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332a08b-37f3-47f3-b0e6-9bb804384967 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:47.240603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:47.246545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:47.249915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:47.281767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:47.282876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:47.283909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:47.283909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:47.283909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:47.283909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:47.283909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:47.283909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:47.283909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:47.283909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:47.285186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 288/581] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:47.285626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:47.421601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332a08b-37f3-47f3-b0e6-9bb804384967 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:47.467998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a98a-a46a-4315-804f-877b018e4b0f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:47.484009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:47.485151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:47.485649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:47.486293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:47.487502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:47.488064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:47.491161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:47.516879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 294/582] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:47.516879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:47.537668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:47.539002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:47.539540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:47.540031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:47.559865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:47.562535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:47.596494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:47.597463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:47.598337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:47.598337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:47.598337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:47.598337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:47.603570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 289/583] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:47.604483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:47.605560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:47.666125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:47.705703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2fc44-2dd8-49fe-a758-79e18f102209 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:47.764564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57bc75-2c3b-4865-93d8-3c041a83aa17 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:47.777797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe105a-0fb6-46b0-8f47-3abc99813648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:47.779391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe105a-0fb6-46b0-8f47-3abc99813648 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:47.780039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe105a-0fb6-46b0-8f47-3abc99813648 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:47.780529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe105a-0fb6-46b0-8f47-3abc99813648 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:47.803755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:47.804417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:47.804944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:47.805225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:47.806027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:47.806345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:47.807413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe105a-0fb6-46b0-8f47-3abc99813648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:47.807836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:47.829601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 290/584] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-67fe105a-0fb6-46b0-8f47-3abc99813648 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:47.829857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:47.831537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:47.832801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:47.833424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:47.833495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 295/585] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:47.833495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe105a-0fb6-46b0-8f47-3abc99813648 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:47.986882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe105a-0fb6-46b0-8f47-3abc99813648 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:47.989881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33afb6-c126-4c46-bffb-2fc20d984285 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:47.992921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:48.000531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:48.037847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:48.038688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.039142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.039650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.040801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:48.041366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:48.042223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:48.043497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.043817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:48.043976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.044351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.056352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:48.058105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:48.071705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 296/586] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:48.071705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:48.074168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:48.075888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 291/587] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:48.079272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:48.219679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af2470-4fcb-412c-8432-a2f76336473b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:48.220816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ed17a-10e2-40c5-9d3b-5915748949ad None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:48.276042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df0c25-1485-4360-903e-1d0aca1d3e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:48.276412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:48.276997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.277274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.277441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0c25-1485-4360-903e-1d0aca1d3e57 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.277740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.277928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0c25-1485-4360-903e-1d0aca1d3e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.278398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0c25-1485-4360-903e-1d0aca1d3e57 None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/dbd31b58571b476bad12735ff7445d55/roles/d31dcaade43b4d03a991cb76b4c830af` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.279017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:48.279770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:48.283055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:48.295260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 297/588] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:48.296451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:48.312321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df0c25-1485-4360-903e-1d0aca1d3e57 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=dbd31b58571b476bad12735ff7445d55, role_id=d31dcaade43b4d03a991cb76b4c830af)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:48.313816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df0c25-1485-4360-903e-1d0aca1d3e57 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:48.354367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:48.394049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 292/589] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:43:48 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/dbd31b58571b476bad12735ff7445d55/roles/d31dcaade43b4d03a991cb76b4c830af => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:48.396474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df0c25-1485-4360-903e-1d0aca1d3e57 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:48.433260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:48.508532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d60221-b7a9-4d5f-97df-ae505b3fd7a2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:48.570863 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:48.571848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.572279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.572682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.583823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:48.585862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:48.607321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:48.609251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 03:43:48.611053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:48.612441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 298/590] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:48.614086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:48.621081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df0c25-1485-4360-903e-1d0aca1d3e57 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:48.674589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f145093-789e-4c54-8369-5e4a96155a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:48.676426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f145093-789e-4c54-8369-5e4a96155a5b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.676426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f145093-789e-4c54-8369-5e4a96155a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.676426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f145093-789e-4c54-8369-5e4a96155a5b None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.677456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f145093-789e-4c54-8369-5e4a96155a5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:48.678121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f145093-789e-4c54-8369-5e4a96155a5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:48.679977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f145093-789e-4c54-8369-5e4a96155a5b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:48.704183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 293/591] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:48.706987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c362-d819-44bd-a4f4-a7a2cea935a7 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.708060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c362-d819-44bd-a4f4-a7a2cea935a7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.708830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c362-d819-44bd-a4f4-a7a2cea935a7 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.712232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 294/592] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:48.713800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499c362-d819-44bd-a4f4-a7a2cea935a7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:48.768876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:48.788994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea965d-e79d-41c7-8e9a-d3d882b9fbc0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:48.824219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:48.833746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:48.835390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.835390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.835390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.848020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:48.850063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:48.856837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499c362-d819-44bd-a4f4-a7a2cea935a7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:48.872211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:48.873770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:48.875406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:48.875406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:48.875406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:48.875406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:48.875406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 299/593] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:48.875406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:48.902608 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc4507-5a86-433c-bcd7-1ab1bd9b4c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:48.903379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4507-5a86-433c-bcd7-1ab1bd9b4c8a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.903653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4507-5a86-433c-bcd7-1ab1bd9b4c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.903928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4507-5a86-433c-bcd7-1ab1bd9b4c8a None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.904786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4507-5a86-433c-bcd7-1ab1bd9b4c8a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:48.905101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4507-5a86-433c-bcd7-1ab1bd9b4c8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:48.906560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4507-5a86-433c-bcd7-1ab1bd9b4c8a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:48.930477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 295/594] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 1 03:43:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:48.931835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a762-6dfa-40ed-b7d0-e91b7d006c6e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:48.932114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a762-6dfa-40ed-b7d0-e91b7d006c6e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:48.932392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a762-6dfa-40ed-b7d0-e91b7d006c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:48.949889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c733a762-6dfa-40ed-b7d0-e91b7d006c6e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:48.951113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c733a762-6dfa-40ed-b7d0-e91b7d006c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:49.041983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3c8ad-5028-4bfb-bcdd-1aec6442c95b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:49.054393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 296/595] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:49.056518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733a762-6dfa-40ed-b7d0-e91b7d006c6e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:49.102196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537b3a-4e7e-4ac8-8b27-3959f0d39081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:49.103645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537b3a-4e7e-4ac8-8b27-3959f0d39081 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:49.104140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537b3a-4e7e-4ac8-8b27-3959f0d39081 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:49.104651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537b3a-4e7e-4ac8-8b27-3959f0d39081 None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/e8327d9916424843b2a67cbdd856ad37/roles/d31dcaade43b4d03a991cb76b4c830af` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:49.119879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:49.130997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77537b3a-4e7e-4ac8-8b27-3959f0d39081 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=e8327d9916424843b2a67cbdd856ad37, role_id=d31dcaade43b4d03a991cb76b4c830af)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:49.133274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77537b3a-4e7e-4ac8-8b27-3959f0d39081 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:49.160260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:49.202179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733a762-6dfa-40ed-b7d0-e91b7d006c6e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:49.224163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 300/596] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:43:48 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/e8327d9916424843b2a67cbdd856ad37/roles/d31dcaade43b4d03a991cb76b4c830af => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:49.225195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537b3a-4e7e-4ac8-8b27-3959f0d39081 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:49.264103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:49.265663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:49.266456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:49.267135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:49.271096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:49.272062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:49.276955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:49.313891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 297/597] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:49.316148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:49.450538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537b3a-4e7e-4ac8-8b27-3959f0d39081 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:49.501474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:49.502689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:49.503200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:49.503712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:49.505280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aa68d-7f9f-47ec-b7ea-a032c4d685fe None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:49.517922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:49.520929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:49.542575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:49.546288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:49.547172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:49.548111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:49.548111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:49.548111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:49.548111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:49.555535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 301/598] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:49.555535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:49.571513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7a77-0b98-4f61-ba32-9a5b4400e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:49.573943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7a77-0b98-4f61-ba32-9a5b4400e68f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:49.574479 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7a77-0b98-4f61-ba32-9a5b4400e68f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:49.575345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7a77-0b98-4f61-ba32-9a5b4400e68f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:49.597404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c7a77-0b98-4f61-ba32-9a5b4400e68f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0f1c7a77-0b98-4f61-ba32-9a5b4400e68f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:49.606153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:49.607541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:49.608627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:49.612515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 298/599] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:49.613576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7a77-0b98-4f61-ba32-9a5b4400e68f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:49.627081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:49.712199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f720ea-831a-48d6-90b2-dde4f86c2c8e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:49.759912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:49.761400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:49.762545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:49.763216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:49.767707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7a77-0b98-4f61-ba32-9a5b4400e68f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:49.779147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:49.780970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:49.800242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:49.801702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 03:43:49.802633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:49.805816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 302/600] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:49.806920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:49.838076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:49.839347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:49.840181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:49.840986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:49.842390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:49.843076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:49.845892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:49.877183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 299/601] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:49.877183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:49.945917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:49.949196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94086-c49f-4f1e-bc00-4499299c4433 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:49.950474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:49.994713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:49.995468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:49.995844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:49.996741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:49.997956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:49.998484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:50.000876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:50.012231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 303/602] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:50.013221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:50.062848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870ebab-e915-48f8-8275-0f2b0ce0e901 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:50.142501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:50.143185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.143489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.143765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.144622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:50.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:50.146446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:50.152862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bd5d7-87f1-4993-95fc-b872e0a75a54 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:50.171276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 300/603] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Jul 1 03:43:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:50.172551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:50.204781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:50.206006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.206006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.206183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.214268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:50.215779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:50.231898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:50.232866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:50.233798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:50.233798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:50.233798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:50.233798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:50.233798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:50.233798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:50.233798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:50.233798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:50.239417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 304/604] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:50.240118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:50.286680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:50.329420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:50.334045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47f964-2506-4824-9077-48bb29ad1594 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:50.396700 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:50.398301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.398896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.399524 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.419659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:50.421811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9665e-5e6c-4811-9b9b-7dec180c50de None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:50.422278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:50.457283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:50.458292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:50.459177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:50.459177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:50.459177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:50.459177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:50.460603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 301/605] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:50.462366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:50.468647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:50.470867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.471414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.471898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.486207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:50.488531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:50.512761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:50.513721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:50.514663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:50.514663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:50.514663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:50.514663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:50.516964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 305/606] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:50.518021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:50.620020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:50.656586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd0e6c-1b11-427b-a041-7b65a6e712e0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:50.664114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29418f56-a07c-47a9-b008-75bd20bb291d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:50.703177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:50.704131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.704566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.704986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.705966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:50.706432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:50.708136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:50.723842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 302/607] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:50.724726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:50.742586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:50.746871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0c7c0-0ddf-486e-8d43-865b3e002647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:50.748079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c7c0-0ddf-486e-8d43-865b3e002647 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.748579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c7c0-0ddf-486e-8d43-865b3e002647 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.749076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c7c0-0ddf-486e-8d43-865b3e002647 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.750572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0c7c0-0ddf-486e-8d43-865b3e002647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:50.751024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0c7c0-0ddf-486e-8d43-865b3e002647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:50.753806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0c7c0-0ddf-486e-8d43-865b3e002647 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:50.782173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 306/608] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:50.782173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ea242-feb1-4dfe-a317-805910309842 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.782173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ea242-feb1-4dfe-a317-805910309842 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.782173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ea242-feb1-4dfe-a317-805910309842 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.788286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 307/609] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:50.788286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ea242-feb1-4dfe-a317-805910309842 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:50.846177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4de15d-704b-436f-a64e-f5eb1a051136 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:50.889011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:50.889576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.889867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.890201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.890926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:50.891311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:50.892815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:50.912778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 303/610] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:50.912778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:50.943988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ea242-feb1-4dfe-a317-805910309842 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:50.986877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a451d-c277-4e78-a076-91636025847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:50.987916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a451d-c277-4e78-a076-91636025847a None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:50.988207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a451d-c277-4e78-a076-91636025847a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:50.988463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a451d-c277-4e78-a076-91636025847a None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/ec8a755e84f04c9590c9c8d57781d2a8/roles/d31dcaade43b4d03a991cb76b4c830af` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:50.992289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:51.005432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:51.008528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a451d-c277-4e78-a076-91636025847a None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=ec8a755e84f04c9590c9c8d57781d2a8, role_id=d31dcaade43b4d03a991cb76b4c830af)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:51.011085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a451d-c277-4e78-a076-91636025847a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:51.043021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61e7a5-5437-407e-b531-943cd842e9d6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:51.078990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3414-091d-4c01-9a1d-933d0b8f5ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:51.079822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3414-091d-4c01-9a1d-933d0b8f5ddb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:51.080104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3414-091d-4c01-9a1d-933d0b8f5ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:51.080380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3414-091d-4c01-9a1d-933d0b8f5ddb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:51.088992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a3414-091d-4c01-9a1d-933d0b8f5ddb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:51.090516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a3414-091d-4c01-9a1d-933d0b8f5ddb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d59a3414-091d-4c01-9a1d-933d0b8f5ddb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:51.106632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:51.108034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 304/611] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:43:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17906f-a371-4ae4-b998-e0463d6595a3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:51.109918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17906f-a371-4ae4-b998-e0463d6595a3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:51.111309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17906f-a371-4ae4-b998-e0463d6595a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:51.140987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 308/612] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:43:50 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/ec8a755e84f04c9590c9c8d57781d2a8/roles/d31dcaade43b4d03a991cb76b4c830af => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:51.140987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a451d-c277-4e78-a076-91636025847a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:51.141324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6d17906f-a371-4ae4-b998-e0463d6595a3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:51.141324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6d17906f-a371-4ae4-b998-e0463d6595a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:51.274583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 305/613] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:51.276134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17906f-a371-4ae4-b998-e0463d6595a3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:51.373042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a451d-c277-4e78-a076-91636025847a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:51.399736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:51.444983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:51.446382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:51.446935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:51.447509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:51.461415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:51.464220 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:51.488834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17906f-a371-4ae4-b998-e0463d6595a3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:51.492557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:51.494505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:51.496560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:51.496560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:51.496560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:51.496560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:51.496560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:51.496560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:51.496560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:51.496560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:51.497397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 309/614] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:51.498226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:51.574580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:51.586129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:51.587079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:51.587556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:51.589605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:51.589605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:51.589605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:51.591191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:51.619638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 306/615] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 1 03:43:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:51.620876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:51.689314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f1e1-813d-4efa-96f6-1ed86ce1776d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:51.738033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dfa7f-0504-43f5-9bd5-388c89fd79a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:51.739153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dfa7f-0504-43f5-9bd5-388c89fd79a1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:51.739703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dfa7f-0504-43f5-9bd5-388c89fd79a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:51.740128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dfa7f-0504-43f5-9bd5-388c89fd79a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:51.753632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5dfa7f-0504-43f5-9bd5-388c89fd79a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4a5dfa7f-0504-43f5-9bd5-388c89fd79a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:51.762386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:51.763547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:51.764380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:51.766833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 310/616] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:51.767899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dfa7f-0504-43f5-9bd5-388c89fd79a1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:51.816308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a78e8-a60f-4e8c-a7c9-834f3eb4b0e1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:51.843034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:51.860494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:51.861939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:51.862370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:51.862791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:51.863807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:51.864320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:51.866042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:51.892469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 307/617] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:51.894288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:51.960202 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dfa7f-0504-43f5-9bd5-388c89fd79a1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:51.963747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:52.001607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:52.002458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.002711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.002965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.015376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:52.016853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:52.037525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:52.039287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:52.041060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:52.041060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:52.041060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:52.041060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:52.043662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 311/618] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:52.044595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:52.119790 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90cb40-73dd-4da6-aba3-664551257dd9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:52.157628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf6b6d-867a-450e-b96a-c6a091d68098 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:52.193168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:52.194498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.195090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.199254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.199254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:52.199654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0a8b6-184d-4b91-9e58-1003cf4a5fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:52.199809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:52.199948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0a8b6-184d-4b91-9e58-1003cf4a5fa9 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.199948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0a8b6-184d-4b91-9e58-1003cf4a5fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.199948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0a8b6-184d-4b91-9e58-1003cf4a5fa9 None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/0beff646206e446386d91537df9eaa67/roles/d31dcaade43b4d03a991cb76b4c830af` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.200863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:52.216880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0a8b6-184d-4b91-9e58-1003cf4a5fa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=0beff646206e446386d91537df9eaa67, role_id=d31dcaade43b4d03a991cb76b4c830af)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:52.218279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0a8b6-184d-4b91-9e58-1003cf4a5fa9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:52.218581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 308/619] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:52.220595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:52.303995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 312/620] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:43:52 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/0beff646206e446386d91537df9eaa67/roles/d31dcaade43b4d03a991cb76b4c830af => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:52.306354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0a8b6-184d-4b91-9e58-1003cf4a5fa9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:52.315923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:52.340567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:52.480474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e1567-5a8c-4ed1-b2be-e8fefea0212e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:52.560668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:52.562217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.562950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.563718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.565307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:52.566210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:52.575264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:52.607696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 309/621] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:52.607970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0a8b6-184d-4b91-9e58-1003cf4a5fa9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:52.617761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:52.669521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:52.670590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.670590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.670886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.696561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:52.698196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:52.721128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:52.722839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:52.724662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:52.724662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:52.724662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:52.724662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:52.724662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 313/622] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:52.728254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:52.818513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ff51f-bf83-4f8a-98c2-a61679c95875 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:52.821024 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:52.821794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:52.866079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadc2a0-4528-44aa-b7e2-9fa9632366ea None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:52.869835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606efb9-7c16-4b6d-b31c-f6185eaf8cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:52.869835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606efb9-7c16-4b6d-b31c-f6185eaf8cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.870467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606efb9-7c16-4b6d-b31c-f6185eaf8cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.870467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606efb9-7c16-4b6d-b31c-f6185eaf8cc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.880087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606efb9-7c16-4b6d-b31c-f6185eaf8cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:52.882496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606efb9-7c16-4b6d-b31c-f6185eaf8cc9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:52.913046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ae0ed-e6c8-470a-9748-3b29a660f290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:52.914443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ae0ed-e6c8-470a-9748-3b29a660f290 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.914952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ae0ed-e6c8-470a-9748-3b29a660f290 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.915401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ae0ed-e6c8-470a-9748-3b29a660f290 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.916324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ae0ed-e6c8-470a-9748-3b29a660f290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:52.916802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ae0ed-e6c8-470a-9748-3b29a660f290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:52.918366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ae0ed-e6c8-470a-9748-3b29a660f290 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2606efb9-7c16-4b6d-b31c-f6185eaf8cc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:52.924889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:52.926665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:52.927814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:52.927814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:52.927814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:52.927814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:52.927814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:52.927814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:52.927814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:52.927814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:52.933233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 310/623] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:52.934948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cd1de-5945-48dd-a562-dcd4c935639c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.935902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cd1de-5945-48dd-a562-dcd4c935639c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.936315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cd1de-5945-48dd-a562-dcd4c935639c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.943360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 311/624] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:52.945624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cd1de-5945-48dd-a562-dcd4c935639c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:52.947512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 314/625] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:43:52.949148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c869-5025-4f32-b005-0515f38c2048 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:52.949606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c869-5025-4f32-b005-0515f38c2048 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:52.950046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c869-5025-4f32-b005-0515f38c2048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:52.976573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-60a4c869-5025-4f32-b005-0515f38c2048 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:52.978559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-60a4c869-5025-4f32-b005-0515f38c2048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:53.065240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 315/626] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:53.066206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4c869-5025-4f32-b005-0515f38c2048 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:53.117832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cd1de-5945-48dd-a562-dcd4c935639c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:53.121716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:53.180710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4c869-5025-4f32-b005-0515f38c2048 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:53.184245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:53.185470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:53.185990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:53.186540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:53.188112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:53.188771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:53.191660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:53.219331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 312/627] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:53.220312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:53.230052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:53.230856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:53.231137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:53.231449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:53.245479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:53.247008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:53.263694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:53.265447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 316/628] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:43:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:53.268317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:53.276101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:53.389837 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d908d-6f5f-4941-b2f1-a589013b1bcf None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:53.400814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:53.423124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec193fc-38a3-4bbf-8c95-4ae9fbec1031 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:53.431889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4afe75-b1eb-4cf4-a97d-ab43c8b08cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:53.432726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afe75-b1eb-4cf4-a97d-ab43c8b08cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:53.433046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afe75-b1eb-4cf4-a97d-ab43c8b08cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:53.433298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afe75-b1eb-4cf4-a97d-ab43c8b08cd0 None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/1123437cabbd4aa09ca3b1a0996a8bfc/roles/d31dcaade43b4d03a991cb76b4c830af` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:53.451379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4afe75-b1eb-4cf4-a97d-ab43c8b08cd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=1123437cabbd4aa09ca3b1a0996a8bfc, role_id=d31dcaade43b4d03a991cb76b4c830af)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:53.452853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4afe75-b1eb-4cf4-a97d-ab43c8b08cd0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:53.493220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa6d69-87de-4da8-90be-7fbded28f111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:53.494513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6d69-87de-4da8-90be-7fbded28f111 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:53.495040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6d69-87de-4da8-90be-7fbded28f111 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:53.495531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6d69-87de-4da8-90be-7fbded28f111 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:53.516182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa6d69-87de-4da8-90be-7fbded28f111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:53.526123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 317/629] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:43:53 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/1123437cabbd4aa09ca3b1a0996a8bfc/roles/d31dcaade43b4d03a991cb76b4c830af => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:43:53.527547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4afe75-b1eb-4cf4-a97d-ab43c8b08cd0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d8fa6d69-87de-4da8-90be-7fbded28f111 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:53.535497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:53.538032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:53.539912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 313/630] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:53.544569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa6d69-87de-4da8-90be-7fbded28f111 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:53.729861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:53.769097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4afe75-b1eb-4cf4-a97d-ab43c8b08cd0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:53.776233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa6d69-87de-4da8-90be-7fbded28f111 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:53.834151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:53.835388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:53.836027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:53.836348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:53.837394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:53.837723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:53.840817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:53.845914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:53.846915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:53.847421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:53.847892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:53.849289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:53.849641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:53.851666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:53.864581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 314/631] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:53.872388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:53.876732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 318/632] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:43:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:53.901773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:53.939123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:54.028508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ea1be-b791-47c5-b1e1-3d40d0ebb183 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:54.065974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5163-a022-40ef-bf81-b625ef3475ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:54.067462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5163-a022-40ef-bf81-b625ef3475ea None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.067749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5163-a022-40ef-bf81-b625ef3475ea None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.068024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5163-a022-40ef-bf81-b625ef3475ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.070530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c8ba5-be42-484c-8af4-f1acccb1382c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:54.123603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cf125-59be-430c-a747-29bca79dcf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:54.124464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf125-59be-430c-a747-29bca79dcf5a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.124744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf125-59be-430c-a747-29bca79dcf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.125017 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf125-59be-430c-a747-29bca79dcf5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.140104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cf125-59be-430c-a747-29bca79dcf5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:54.142091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cf125-59be-430c-a747-29bca79dcf5a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:54.146263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-796cf125-59be-430c-a747-29bca79dcf5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:54.165599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:54.166698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:54.167721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:54.167721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:54.167721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:54.167721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:54.174824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 315/633] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:54.174824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7c3e-0338-437f-bb59-7870eff975cf None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.174824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7c3e-0338-437f-bb59-7870eff975cf None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.175463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7c3e-0338-437f-bb59-7870eff975cf None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.187105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 316/634] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:54.188647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb7c3e-0338-437f-bb59-7870eff975cf None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:54.194075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc5163-a022-40ef-bf81-b625ef3475ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:54.196864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc5163-a022-40ef-bf81-b625ef3475ea None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:54.218869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:54.328766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 319/635] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:54.331205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ad2a-af95-4f5d-91bd-4912c6550b70 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.332051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ad2a-af95-4f5d-91bd-4912c6550b70 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.332587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ad2a-af95-4f5d-91bd-4912c6550b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.345854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb7c3e-0338-437f-bb59-7870eff975cf None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:54.356641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-bb46ad2a-af95-4f5d-91bd-4912c6550b70 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:54.358833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bb46ad2a-af95-4f5d-91bd-4912c6550b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:54.388009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:54.389488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.389774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.390065 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.391181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:54.391588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:54.393280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:54.415268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 317/636] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:54.415268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:54.458583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 320/637] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:54.459996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46ad2a-af95-4f5d-91bd-4912c6550b70 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:54.513618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:54.531935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:54.567182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3880-7319-47cb-a808-1450d57acccb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:54.596252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46ad2a-af95-4f5d-91bd-4912c6550b70 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:54.607884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:54.609067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.609584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.610056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.623369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:54.625222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:54.637975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:54.639194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.639528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.639803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] PATH_INFO: `/v3/users/glance` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:54.646817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:54.647826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:54.648780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:54.648780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:54.648780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:54.648780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:54.648997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:54.650037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:54.651680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 318/638] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:54.652850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:54.666880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:54.667934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 03:43:54.669472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:54.672954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 321/639] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:54.674025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:54.770718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1502a3a-9356-4c2e-ba33-fbc0b4474f00 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:54.789655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:54.796948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358d776-7f0a-47c0-808c-d6025e5dd6ea None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:54.809877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:54.838696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089974c-87b5-477a-b4e8-2527d4e16ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:54.843529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:54.843529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.843752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089974c-87b5-477a-b4e8-2527d4e16ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:54.843828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.843828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.843974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089974c-87b5-477a-b4e8-2527d4e16ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:54.843974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089974c-87b5-477a-b4e8-2527d4e16ab9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:54.844099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:54.844099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:54.844099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:54.858035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 322/640] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:54.859258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:54.864020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089974c-87b5-477a-b4e8-2527d4e16ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b089974c-87b5-477a-b4e8-2527d4e16ab9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:54.880349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:54.881873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:54.883217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:54.884268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 319/641] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:54.885080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089974c-87b5-477a-b4e8-2527d4e16ab9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:55.001453 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e8e4b-16fb-433e-8bc1-9cddc87f6686 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:55.040991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089974c-87b5-477a-b4e8-2527d4e16ab9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:55.067772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874acfd-861d-449b-96cf-817de89764ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:55.068939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874acfd-861d-449b-96cf-817de89764ff None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.069441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874acfd-861d-449b-96cf-817de89764ff None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.069932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874acfd-861d-449b-96cf-817de89764ff None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.071612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874acfd-861d-449b-96cf-817de89764ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:55.072164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874acfd-861d-449b-96cf-817de89764ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:55.074941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874acfd-861d-449b-96cf-817de89764ff None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:55.085389 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:55.086201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.086521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.086787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.096014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:55.097562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:55.115393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 323/642] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 1 03:43:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:55.115393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3a57-d635-4ef1-be9c-b8f4a8e77539 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.115393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3a57-d635-4ef1-be9c-b8f4a8e77539 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:55.115886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:55.117478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:55.124779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:55.124779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:55.124779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:55.124779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:55.124779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:55.124779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:55.124779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:55.124779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:55.125834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3a57-d635-4ef1-be9c-b8f4a8e77539 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.125834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 324/643] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:55.125834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb3a57-d635-4ef1-be9c-b8f4a8e77539 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:55.126264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 320/644] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:55.126264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:55.168946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:55.178602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:55.269911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955ff7e-db0b-469c-8aa4-9980646a1b0d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:55.281389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb3a57-d635-4ef1-be9c-b8f4a8e77539 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:55.318456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:55.319124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.319442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.319743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.320562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:55.320834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:55.322309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:55.324935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efcde1-ec5e-4972-8de7-0290e04e0c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:55.325957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efcde1-ec5e-4972-8de7-0290e04e0c97 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.326414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efcde1-ec5e-4972-8de7-0290e04e0c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.326815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efcde1-ec5e-4972-8de7-0290e04e0c97 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.327898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efcde1-ec5e-4972-8de7-0290e04e0c97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:55.328334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efcde1-ec5e-4972-8de7-0290e04e0c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:55.330723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efcde1-ec5e-4972-8de7-0290e04e0c97 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:55.337276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 321/645] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:55.342898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:55.347270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 325/646] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:55.349436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e1f50-5e52-4f48-aac6-059a79af5dbe None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.349892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e1f50-5e52-4f48-aac6-059a79af5dbe None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.350332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e1f50-5e52-4f48-aac6-059a79af5dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.371351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-573e1f50-5e52-4f48-aac6-059a79af5dbe None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:55.372830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-573e1f50-5e52-4f48-aac6-059a79af5dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:55.452876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:55.464278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 326/647] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:55.465742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e1f50-5e52-4f48-aac6-059a79af5dbe None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:55.481884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa104b-e788-407e-b884-ea15fc4c00a3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:55.492218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:55.532955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1145a-0f19-4166-8d58-57327404c84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:55.533878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1145a-0f19-4166-8d58-57327404c84b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.534313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1145a-0f19-4166-8d58-57327404c84b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.534883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1145a-0f19-4166-8d58-57327404c84b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.619446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e1f50-5e52-4f48-aac6-059a79af5dbe None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:55.648999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1145a-0f19-4166-8d58-57327404c84b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:55.650539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1145a-0f19-4166-8d58-57327404c84b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:55.667689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:55.668371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.668665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.669011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.669882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=dbd31b58571b476bad12735ff7445d55) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:55.670256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:55.671897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:55.691272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 327/648] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:55 2022] GET /identity/v3/role_assignments?user.id=dbd31b58571b476bad12735ff7445d55&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:55.691272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:55.831948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:55.850637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:55.857606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 322/649] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:55.859990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1145a-0f19-4166-8d58-57327404c84b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:55.885687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95bc4a-17c4-4b79-aeb8-1dbeb29f045b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:55.953395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:55.955336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:55.956158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:55.956964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:55.980998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:55.984222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:55.987611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1145a-0f19-4166-8d58-57327404c84b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:56.019435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:56.020942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:56.021833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:56.021833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:56.021833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:56.021833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:56.027486 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 328/650] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:56.029061 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:56.037756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65944b56-6257-4f36-b1b4-4da3ace713d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:56.039035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65944b56-6257-4f36-b1b4-4da3ace713d7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:56.040530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65944b56-6257-4f36-b1b4-4da3ace713d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:56.041049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65944b56-6257-4f36-b1b4-4da3ace713d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:56.065107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65944b56-6257-4f36-b1b4-4da3ace713d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-65944b56-6257-4f36-b1b4-4da3ace713d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:56.075945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:56.077621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:56.079168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:56.084847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 323/651] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:56.086480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65944b56-6257-4f36-b1b4-4da3ace713d7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:56.198962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:56.252419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91722e7-47e0-4015-adbf-def8d001e00e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:56.314256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65944b56-6257-4f36-b1b4-4da3ace713d7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:56.320828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:56.331374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:56.333066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:56.338639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:56.338639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:56.360712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:56.364850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:56.385048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93532bb-b603-4db2-9295-b11caf41a034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:56.386222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93532bb-b603-4db2-9295-b11caf41a034 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:56.386724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93532bb-b603-4db2-9295-b11caf41a034 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:56.387257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93532bb-b603-4db2-9295-b11caf41a034 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:56.388629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93532bb-b603-4db2-9295-b11caf41a034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:56.389182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93532bb-b603-4db2-9295-b11caf41a034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:56.391079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93532bb-b603-4db2-9295-b11caf41a034 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:56.400353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:56.401319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:56.402208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:56.402208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:56.402208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:56.402208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:56.407310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 329/652] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:56.407310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:56.417707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 324/653] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:56.420141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3dc8c-2071-42f9-8e6d-8e5dde074447 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:56.420640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3dc8c-2071-42f9-8e6d-8e5dde074447 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:56.421124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3dc8c-2071-42f9-8e6d-8e5dde074447 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:56.424621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 325/654] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:56.425589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3dc8c-2071-42f9-8e6d-8e5dde074447 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:56.578131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3dc8c-2071-42f9-8e6d-8e5dde074447 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:56.585883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd0b4e-7842-431b-987c-def9dbb404a8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:56.590984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:56.622435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:56.623397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:56.623810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:56.624213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:56.625084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:56.625528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:56.627471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:56.650275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 326/655] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:56.650275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:56.661189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83988fc1-102a-47ef-8b86-167dea389d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:56.662254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988fc1-102a-47ef-8b86-167dea389d1e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:56.662711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988fc1-102a-47ef-8b86-167dea389d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:56.663163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988fc1-102a-47ef-8b86-167dea389d1e None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:56.664320 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83988fc1-102a-47ef-8b86-167dea389d1e None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:56.664825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83988fc1-102a-47ef-8b86-167dea389d1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:56.667177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83988fc1-102a-47ef-8b86-167dea389d1e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:56.681186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 330/656] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:56.689716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710af50-b67d-466e-bc73-613b83ca2569 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:56.690759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710af50-b67d-466e-bc73-613b83ca2569 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:56.691260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710af50-b67d-466e-bc73-613b83ca2569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:56.721239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5710af50-b67d-466e-bc73-613b83ca2569 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:56.724023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5710af50-b67d-466e-bc73-613b83ca2569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:56.798331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:56.802077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58f262-e8a3-45e7-816b-c843ba14dee2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:56.838723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 331/657] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:56.839818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710af50-b67d-466e-bc73-613b83ca2569 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:56.858594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:56.860528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:56.861027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:56.861517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:56.876518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:56.880740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:56.911168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:56.913130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 03:43:56.915281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:56.918952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 327/658] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:56.920410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:56.964229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:57.002531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710af50-b67d-466e-bc73-613b83ca2569 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.043625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.044416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.044693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.044971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.048280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb414c4-8ccf-4553-84b4-a280ae31daf6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.052254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:57.053711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:57.068663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:57.069630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:57.070529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 332/659] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:57.071446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:57.098581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.099303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.099585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.099855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.111043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:57.119310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:57.132101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:57.140314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:57.141259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:57.155086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:57.155086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:57.155086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:57.155086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:57.155086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 328/660] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:57.155086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:57.224916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab12ec-d02c-49c0-aacd-51c2b4172ac6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.250826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:57.274987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627eaae-9710-4770-b9fd-6127ad827448 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.278121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06157843-c819-468d-9f2a-d250cc3bdb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.278932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06157843-c819-468d-9f2a-d250cc3bdb06 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.279241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06157843-c819-468d-9f2a-d250cc3bdb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.279520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06157843-c819-468d-9f2a-d250cc3bdb06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.293399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06157843-c819-468d-9f2a-d250cc3bdb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-06157843-c819-468d-9f2a-d250cc3bdb06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:57.301816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:57.302814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:57.303722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 333/661] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:57.304422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06157843-c819-468d-9f2a-d250cc3bdb06 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:57.338629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.340596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.341463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.341967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.344715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:57.345291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:57.349859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:57.375767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 329/662] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 1 03:43:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:57.376933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:57.438189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:57.446232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06157843-c819-468d-9f2a-d250cc3bdb06 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.484128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.484790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.485084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.485364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.486142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:57.486442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:57.487922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:57.507275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 334/663] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:57.507275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:57.541503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a849e-2b10-4de2-a2f1-7003b0e229a6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.590818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.591522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.591810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.592088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.592813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:57.593185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:57.594565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:57.602779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:57.612939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 330/664] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:57.612939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:57.659411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09c138-8573-4eed-9899-ce37220931b1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.706969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.707773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.708150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.708504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.709344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:57.709723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:57.711214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:57.728123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 335/665] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:43:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:57.728123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:57.752784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:57.772233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ccd53-b57a-409e-8991-9560f1e9aa1a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.815722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9466e-58a2-4200-81e9-b60e6cdf9b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.816382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9466e-58a2-4200-81e9-b60e6cdf9b02 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.816755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9466e-58a2-4200-81e9-b60e6cdf9b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.817048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9466e-58a2-4200-81e9-b60e6cdf9b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.878317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40770ddf-e6cd-4339-b7b2-4641f2721ad1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:57.922594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:57.929862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9466e-58a2-4200-81e9-b60e6cdf9b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:57.932322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9466e-58a2-4200-81e9-b60e6cdf9b02 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:57.937088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:57.938029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:57.938497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:57.938953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:57.940450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=e8327d9916424843b2a67cbdd856ad37) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:57.940970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:57.943584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:57.963269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 336/666] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:43:57 2022] GET /identity/v3/role_assignments?user.id=e8327d9916424843b2a67cbdd856ad37&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:57.963269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:58.124560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:58.128955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 331/667] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:58.130533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9466e-58a2-4200-81e9-b60e6cdf9b02 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:58.154826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea4407-d6a4-4d5e-93e1-c67df12a1efc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:58.213847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:58.215504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:58.215785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:58.216866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:58.227550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:58.230864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:58.246807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:58.247994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 337/668] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:43:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:58.249099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:58.298563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:58.303208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9466e-58a2-4200-81e9-b60e6cdf9b02 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:58.355149 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36ea01-ca9b-43a0-a775-d9ebfbab6091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:58.356549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36ea01-ca9b-43a0-a775-d9ebfbab6091 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:58.357052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36ea01-ca9b-43a0-a775-d9ebfbab6091 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:58.357541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36ea01-ca9b-43a0-a775-d9ebfbab6091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:58.374374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36ea01-ca9b-43a0-a775-d9ebfbab6091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:58.375932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36ea01-ca9b-43a0-a775-d9ebfbab6091 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:58.391719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ff219-bf25-4bf7-95d2-750b0c8da2a7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-da36ea01-ca9b-43a0-a775-d9ebfbab6091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:58.402006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:58.402976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 332/669] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1722f8a-a03b-4201-8881-fe7330e2292a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1722f8a-a03b-4201-8881-fe7330e2292a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1722f8a-a03b-4201-8881-fe7330e2292a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:58.409106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 333/670] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:43:58.409603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1722f8a-a03b-4201-8881-fe7330e2292a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:58.448171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:58.449365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:58.449661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:58.449936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:58.462580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:58.464075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:58.466830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:58.483742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:58.484730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:58.486138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:58.486138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:58.486138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:58.486138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:58.486138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 338/671] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:58.486138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:58.558840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1722f8a-a03b-4201-8881-fe7330e2292a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:58.609981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c3b5f-e2fd-4f81-a2dd-faabe745c02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:58.610994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c3b5f-e2fd-4f81-a2dd-faabe745c02b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:58.611304 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c3b5f-e2fd-4f81-a2dd-faabe745c02b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:58.611811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c3b5f-e2fd-4f81-a2dd-faabe745c02b None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:58.612420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c3b5f-e2fd-4f81-a2dd-faabe745c02b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:58.612729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c3b5f-e2fd-4f81-a2dd-faabe745c02b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:58.614327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c3b5f-e2fd-4f81-a2dd-faabe745c02b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:58.616568 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d1a1-bece-4cd3-8d38-085bb03b9d02 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:58.634865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 334/672] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Jul 1 03:43:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:58.636198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb131e-6855-45e3-9110-264ac8f82f28 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:58.636482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb131e-6855-45e3-9110-264ac8f82f28 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:58.636755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb131e-6855-45e3-9110-264ac8f82f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:58.655584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2beb131e-6855-45e3-9110-264ac8f82f28 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:43:58.656704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2beb131e-6855-45e3-9110-264ac8f82f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:43:58.658994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:58.667462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:58.668610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:58.668894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:58.669176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:58.669965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:58.674468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:58.674468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:58.694794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 339/673] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:58.695889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:58.775419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:58.783279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 335/674] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:43:58.783279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb131e-6855-45e3-9110-264ac8f82f28 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:58.838356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182e1da-01c6-45d2-a16d-9b117bb8fb64 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:58.903818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:58.905330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:58.909028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:58.909028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:58.909028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:58.909028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:58.909028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:58.931028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 340/675] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:58.931028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:58.959158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb131e-6855-45e3-9110-264ac8f82f28 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:59.015741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bcb2e-d1f9-4499-8494-e29c453c654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:59.016482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bcb2e-d1f9-4499-8494-e29c453c654d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:59.016674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bcb2e-d1f9-4499-8494-e29c453c654d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:59.016959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bcb2e-d1f9-4499-8494-e29c453c654d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:59.026783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:59.100669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bcb2e-d1f9-4499-8494-e29c453c654d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:59.102153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bcb2e-d1f9-4499-8494-e29c453c654d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:59.110607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716cc1-25dd-453c-825f-11fd331f7444 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:59.135695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:59.170603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:59.171951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:59.172439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:59.173052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:59.187491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:59.190239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:43:59.218754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:59.219824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:43:59.226365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:59.227001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 336/676] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:43:59.227001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bcb2e-d1f9-4499-8494-e29c453c654d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:59.228753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 341/677] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:43:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:59.230200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:59.281876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:59.353713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681904b-193b-4784-bbae-40bd11ce2057 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:59.355372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bcb2e-d1f9-4499-8494-e29c453c654d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:59.398458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:59.399159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcaeac-3bc7-4847-97f3-643586a21060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:59.399730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:59.400185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:59.400361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcaeac-3bc7-4847-97f3-643586a21060 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:59.400643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:59.400797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcaeac-3bc7-4847-97f3-643586a21060 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:59.401235 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcaeac-3bc7-4847-97f3-643586a21060 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:59.413238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:59.415710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:59.419974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcaeac-3bc7-4847-97f3-643586a21060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-dcfcaeac-3bc7-4847-97f3-643586a21060 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:43:59.434912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:59.436125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 337/678] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:43:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:59.444919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcaeac-3bc7-4847-97f3-643586a21060 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:43:59.446463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:59.448321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 342/679] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:43:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:59.453292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:59.505869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:59.601466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:59.608116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcaeac-3bc7-4847-97f3-643586a21060 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:59.611772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2083aeb-17e1-4888-bfa8-7d6db6835c77 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:59.657683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:59.658442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:59.658764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:59.659037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:59.659912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:59.660183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:59.660794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:59.661472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:59.661750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:59.662020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:59.662297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:59.673406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:59.674873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:59.688401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 338/680] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 1 03:43:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:59.688978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:43:59.708051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:43:59.710070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:43:59.712964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:43:59.712964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:43:59.712964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:43:59.712964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:43:59.715461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 343/681] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:43:59.717532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:59.838585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:43:59.842937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710783e-6299-4a37-880d-45834de18dfc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:59.860377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc814034-c612-4186-9b38-2bdfe551e337 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:43:59.896422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:59.897414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:59.897656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:59.897938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:59.898703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:59.899012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:59.900439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:59.904138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:43:59.904138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:43:59.904138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:43:59.904138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:43:59.904668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:43:59.904668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:43:59.904924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:43:59.913622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 339/682] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:43:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:59.915026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:59.925615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 344/683] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:43:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:43:59.927050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:43:59.955468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:00.062774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373756-a2bd-4595-92b5-bcfac667092b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:00.121029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1df3c-f612-4c80-ac43-3f4e9b6f93bc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:00.124781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:00.125727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.126144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.126544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.127871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:00.128396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:00.131385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:00.155144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 345/684] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:00.156655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:00.172819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:00.184518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6a722-1968-4ee8-86ee-074be8598a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:00.190117 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6a722-1968-4ee8-86ee-074be8598a79 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.190117 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6a722-1968-4ee8-86ee-074be8598a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.190117 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6a722-1968-4ee8-86ee-074be8598a79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.288010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6a722-1968-4ee8-86ee-074be8598a79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:00.289477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6a722-1968-4ee8-86ee-074be8598a79 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:00.292418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fab8d-b7d3-41b0-93a6-cb93aba3605c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:00.309643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:00.346115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:00.347457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.348009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.348484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.368252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:00.371130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:00.397073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 340/685] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:44:00.397931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6a722-1968-4ee8-86ee-074be8598a79 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:00.404528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:00.406371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:00.408254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:00.408254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:00.408254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:00.408254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:00.408254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 346/686] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:00.408254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:00.484883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:00.528275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6a722-1968-4ee8-86ee-074be8598a79 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:00.536984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269d5e1-0026-4fa3-ab86-8685bda90d0f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:00.551901 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:00.571772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d94f5d-5e15-425b-9487-47e24ff02bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:00.572535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d94f5d-5e15-425b-9487-47e24ff02bfa None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.572881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d94f5d-5e15-425b-9487-47e24ff02bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.573251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d94f5d-5e15-425b-9487-47e24ff02bfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.574247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d94f5d-5e15-425b-9487-47e24ff02bfa None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=ec8a755e84f04c9590c9c8d57781d2a8) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:00.575654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d94f5d-5e15-425b-9487-47e24ff02bfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:00.577684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d94f5d-5e15-425b-9487-47e24ff02bfa None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:00.584345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:00.585149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.585434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.585710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.586330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 341/687] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:00 2022] GET /identity/v3/role_assignments?user.id=ec8a755e84f04c9590c9c8d57781d2a8&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:00.587834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51cbf1-0496-423e-a0a6-a2ea6bb76ea1 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.588273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51cbf1-0496-423e-a0a6-a2ea6bb76ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.588649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51cbf1-0496-423e-a0a6-a2ea6bb76ea1 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.590160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 342/688] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:00.591043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51cbf1-0496-423e-a0a6-a2ea6bb76ea1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:00.605252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:00.606771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:00.623528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:00.632277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 347/689] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:00.633167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:00.776649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51cbf1-0496-423e-a0a6-a2ea6bb76ea1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:00.820026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29516e6-8a9d-44eb-8006-2ef9bea7cf11 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:00.842809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116badb-626b-469f-b6c7-f316191cab3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:00.844151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116badb-626b-469f-b6c7-f316191cab3a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.845181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116badb-626b-469f-b6c7-f316191cab3a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.845181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116badb-626b-469f-b6c7-f316191cab3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.848744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:00.864203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116badb-626b-469f-b6c7-f316191cab3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:00.866598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116badb-626b-469f-b6c7-f316191cab3a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:00.874799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:00.875909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.876359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.876734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.877731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:00.878217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:00.881434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c116badb-626b-469f-b6c7-f316191cab3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:00.896704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:00.902582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:00.905127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:00.905127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:00.905127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:00.905127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:00.905127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 343/690] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:00.905127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47c0be-2e7f-4dbb-a5e5-15fe91d526ff None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:00.905127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47c0be-2e7f-4dbb-a5e5-15fe91d526ff None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:00.906876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 348/691] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:00.907035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47c0be-2e7f-4dbb-a5e5-15fe91d526ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:00.907186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:00.923824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7a47c0be-2e7f-4dbb-a5e5-15fe91d526ff None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:00.925254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7a47c0be-2e7f-4dbb-a5e5-15fe91d526ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:00.963185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:01.040740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 344/692] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:44:01.043800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47c0be-2e7f-4dbb-a5e5-15fe91d526ff None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:01.087777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d216a6-5c69-4ff9-a269-fa839339b3ed None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:01.158077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:01.160420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:01.161140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:01.161800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:01.163862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:01.165118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:01.167940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:01.199293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 349/693] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:01.199293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:01.236037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47c0be-2e7f-4dbb-a5e5-15fe91d526ff None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:01.253033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:01.290165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:01.291050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:01.291200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:01.291392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:01.292125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:01.292431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:01.293826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:01.309391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 345/694] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:01.310103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:01.347818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:01.367174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8ad61-1440-4c0a-8a4d-5b54acbeba43 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:01.410552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:01.411737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:01.411737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:01.411997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:01.420453 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:01.421962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:01.438633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:01.440698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:01.442765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 350/695] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:44:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:01.445441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:01.483101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b78c86-41a8-43da-85eb-11863f19687e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:01.531551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:01.532901 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:01.533384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:01.533869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:01.547284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:01.548812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:01.571773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:01.581495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:01.583599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:01.583599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:01.583599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:01.583599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:01.583599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 346/696] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:01.583599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:01.628301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:01.633421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf2251-247a-43d3-a409-1148272140ac None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:01.676625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:01.677785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:01.678907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:01.679913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:01.681813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=0beff646206e446386d91537df9eaa67) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:01.682625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:01.685797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:01.698746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 351/697] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:01 2022] GET /identity/v3/role_assignments?user.id=0beff646206e446386d91537df9eaa67&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:01.700326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:01.715867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:01.759994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87713e-8f4a-4f9d-9d9a-4c7943af9520 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:01.826019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:01.827054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:01.827563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:01.828178 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:01.843170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:01.845964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:01.859639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e1bd9-d915-477b-a269-4fb7c7db514a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:01.874492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:01.876786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:01.877692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:01.877692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:01.877692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:01.877692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:01.877692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:01.877692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:01.877692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:01.877692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:01.884611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 347/698] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:01.885880 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:01.908294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:01.909490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:01.910059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:01.911654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:01.912904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:01.913464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:01.916007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:01.947287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 352/699] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:44:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:01.947287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:01.988196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:02.081506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951669-2036-49ef-b4f9-0e722b8be9c7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:02.121580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:02.122258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:02.122542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:02.122812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:02.123776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:02.124086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:02.125476 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:02.133973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:02.150951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 348/700] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:02.150951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:02.163591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3e54-5cd0-4aa4-8a14-cd2dec5a0ad7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:02.252642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:02.253804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:02.254299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:02.254787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:02.257624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:02.258192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:02.262392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:02.282501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dab7b-643d-4988-848c-731139893cb1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:02.282930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 353/701] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:02.284148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:02.353378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fb3aa-b16f-4274-9ab8-3937c00488eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:02.354062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fb3aa-b16f-4274-9ab8-3937c00488eb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:02.354324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fb3aa-b16f-4274-9ab8-3937c00488eb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:02.354595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fb3aa-b16f-4274-9ab8-3937c00488eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:02.390633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:02.423624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6d052-a124-4bf2-bc8d-c742202f7bd4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:02.450230 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:02.462334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fb3aa-b16f-4274-9ab8-3937c00488eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:02.464537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fb3aa-b16f-4274-9ab8-3937c00488eb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:02.487514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403eba-2586-4124-98e3-a5de8fc255dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:02.489138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403eba-2586-4124-98e3-a5de8fc255dc None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:02.489525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403eba-2586-4124-98e3-a5de8fc255dc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:02.489808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403eba-2586-4124-98e3-a5de8fc255dc None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:02.490618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55403eba-2586-4124-98e3-a5de8fc255dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:02.493074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55403eba-2586-4124-98e3-a5de8fc255dc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:02.595568 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 354/702] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:44:02 2022] POST /identity/v3/users => generated 254 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:02.597154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403eba-2586-4124-98e3-a5de8fc255dc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:02.633085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 349/703] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:44:02.642353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fb3aa-b16f-4274-9ab8-3937c00488eb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:02.848048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:02.860816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fb3aa-b16f-4274-9ab8-3937c00488eb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:02.940129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272e7fc-8365-434d-bd8e-31e85d55333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:02.943006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272e7fc-8365-434d-bd8e-31e85d55333f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:02.943486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272e7fc-8365-434d-bd8e-31e85d55333f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:02.943934 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272e7fc-8365-434d-bd8e-31e85d55333f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:02.962328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403eba-2586-4124-98e3-a5de8fc255dc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:02.970314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272e7fc-8365-434d-bd8e-31e85d55333f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:02.973864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272e7fc-8365-434d-bd8e-31e85d55333f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1272e7fc-8365-434d-bd8e-31e85d55333f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:03.009280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:03.010278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:03.011253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:03.011253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:03.011253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:03.011253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:03.015249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 350/704] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:03.015418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc012d-85b0-4509-9f60-ce8cdc57b06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:03.015525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707964-48ec-46a2-9910-b196f02fe3d0 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:03.015614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc012d-85b0-4509-9f60-ce8cdc57b06a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:03.015614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc012d-85b0-4509-9f60-ce8cdc57b06a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:03.015757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707964-48ec-46a2-9910-b196f02fe3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:03.015825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc012d-85b0-4509-9f60-ce8cdc57b06a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:03.015891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707964-48ec-46a2-9910-b196f02fe3d0 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:03.015957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc012d-85b0-4509-9f60-ce8cdc57b06a None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=1123437cabbd4aa09ca3b1a0996a8bfc) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:03.016212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc012d-85b0-4509-9f60-ce8cdc57b06a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:03.017664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc012d-85b0-4509-9f60-ce8cdc57b06a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:03.018136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 351/705] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:03.021084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a707964-48ec-46a2-9910-b196f02fe3d0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:03.026021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:03.033953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 355/706] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:02 2022] GET /identity/v3/role_assignments?user.id=1123437cabbd4aa09ca3b1a0996a8bfc&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 608 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:03.033953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd374a41-32f0-4be8-bfdf-c46bc5511692 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:03.033953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd374a41-32f0-4be8-bfdf-c46bc5511692 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:03.033953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd374a41-32f0-4be8-bfdf-c46bc5511692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:03.074299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-bd374a41-32f0-4be8-bfdf-c46bc5511692 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:03.078618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bd374a41-32f0-4be8-bfdf-c46bc5511692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:03.235181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 356/707] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:44:03.258823 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a707964-48ec-46a2-9910-b196f02fe3d0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:03.296683 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd374a41-32f0-4be8-bfdf-c46bc5511692 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:03.330877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:03.332613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:03.333318 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:03.334038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:03.335631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:03.336564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:03.339289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:03.362711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 352/708] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:03.372596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:03.391398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:03.510560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad233f70-3bb3-4a23-90a1-c1e2d7307a2c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:03.533316 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd374a41-32f0-4be8-bfdf-c46bc5511692 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:03.562222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:03.562222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:03.562671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:03.562807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:03.574445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:03.576022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:03.588393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:03.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:03.608736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:03.610528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:03.610528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:03.610528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:03.610528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:03.610528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:03.610528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:03.610528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:03.610528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:03.611459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:03.611612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 353/709] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:03.611612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:03.611873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:03.612235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:03.612475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:03.613921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:03.615070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:03.634233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:03.636213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:03.639289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 357/710] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Jul 1 03:44:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:03.644300 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:03.755648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:03.795323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d81e94-fd11-4be8-a4da-a1e6c086a153 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:03.815498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26ab35-5794-45bb-9e47-325777109615 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:03.853938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:03.854614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:03.854895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:03.855162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:03.855925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:03.856234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:03.857632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:03.863019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 358/711] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 1 03:44:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:03.869193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:03.925139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:03.926386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:03.926674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:03.926952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:03.927793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:03.928112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:03.930063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:03.961841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 354/712] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:03.967771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:03.975960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:03.984264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de535b46-b3ca-4fc7-a925-64413e8045cc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:04.023173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a5370-65ec-4527-9586-61886b2dac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:04.024022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5370-65ec-4527-9586-61886b2dac96 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:04.024356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5370-65ec-4527-9586-61886b2dac96 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:04.024652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5370-65ec-4527-9586-61886b2dac96 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:04.025386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a5370-65ec-4527-9586-61886b2dac96 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:04.025701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a5370-65ec-4527-9586-61886b2dac96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:04.027111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a5370-65ec-4527-9586-61886b2dac96 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:04.034389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 359/713] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 1 03:44:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:04.146153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324aeda3-750b-4201-aaf3-3306fa6a9156 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:04.178266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:04.187551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef63b30-b71b-40c8-bbaa-71fe72374615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:04.192666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef63b30-b71b-40c8-bbaa-71fe72374615 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:04.192666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef63b30-b71b-40c8-bbaa-71fe72374615 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:04.192666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef63b30-b71b-40c8-bbaa-71fe72374615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:04.192666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef63b30-b71b-40c8-bbaa-71fe72374615 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=dbd31b58571b476bad12735ff7445d55) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:04.192666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef63b30-b71b-40c8-bbaa-71fe72374615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:04.192666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef63b30-b71b-40c8-bbaa-71fe72374615 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:04.198330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 355/714] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:03 2022] GET /identity/v3/role_assignments?user.id=dbd31b58571b476bad12735ff7445d55&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:04.569968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9143c-4920-4e47-a0c3-8d97bb6f2b72 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:04.570757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9143c-4920-4e47-a0c3-8d97bb6f2b72 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:04.571329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9143c-4920-4e47-a0c3-8d97bb6f2b72 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:04.574489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 360/715] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:04.583952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734eab-a10a-4fa4-8717-36e30179030f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:04.584682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734eab-a10a-4fa4-8717-36e30179030f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:04.585256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734eab-a10a-4fa4-8717-36e30179030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:04.615308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1c734eab-a10a-4fa4-8717-36e30179030f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:04.617523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1c734eab-a10a-4fa4-8717-36e30179030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:04.757275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 356/716] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:44:04.794041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9143c-4920-4e47-a0c3-8d97bb6f2b72 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:05.004274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9143c-4920-4e47-a0c3-8d97bb6f2b72 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:05.053355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:05.078372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:05.079761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:05.080254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:05.080739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] PATH_INFO: `/v3/services/network` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:05.082014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:05.084772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:05.113722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:05.115669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:05.117277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 361/717] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:04 2022] GET /identity/v3/services/network => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:05.126091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c734eab-a10a-4fa4-8717-36e30179030f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:05.333934 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c734eab-a10a-4fa4-8717-36e30179030f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:05.380841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:05.403121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf8c77-f6de-48ab-8b0e-7ca0cf0ce974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:05.404320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf8c77-f6de-48ab-8b0e-7ca0cf0ce974 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:05.404814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf8c77-f6de-48ab-8b0e-7ca0cf0ce974 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:05.405305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf8c77-f6de-48ab-8b0e-7ca0cf0ce974 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:05.406698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf8c77-f6de-48ab-8b0e-7ca0cf0ce974 None admin] RBAC: Adding query filter params (name=network) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:05.407268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf8c77-f6de-48ab-8b0e-7ca0cf0ce974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:05.409994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf8c77-f6de-48ab-8b0e-7ca0cf0ce974 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:05.419499 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 357/718] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:05.428454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:05.659194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae270a-4f19-49a2-835c-d0b2b4c7366c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:05.757407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497da7f-92a0-4121-af43-cf5d35f71bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:05.758468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497da7f-92a0-4121-af43-cf5d35f71bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:05.758945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497da7f-92a0-4121-af43-cf5d35f71bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:05.759459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497da7f-92a0-4121-af43-cf5d35f71bd7 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:05.760751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497da7f-92a0-4121-af43-cf5d35f71bd7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:05.761303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497da7f-92a0-4121-af43-cf5d35f71bd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:05.763929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497da7f-92a0-4121-af43-cf5d35f71bd7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:05.772988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 362/719] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:06.329533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccbccc-38dd-4468-842d-06f1db2abd4a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:06.330280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccbccc-38dd-4468-842d-06f1db2abd4a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:06.330615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccbccc-38dd-4468-842d-06f1db2abd4a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:06.334127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 358/720] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:06.342504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743ff44-6231-471e-9b16-98a371fa70cc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:06.342970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743ff44-6231-471e-9b16-98a371fa70cc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:06.343461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743ff44-6231-471e-9b16-98a371fa70cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:06.380052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e743ff44-6231-471e-9b16-98a371fa70cc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:06.382101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e743ff44-6231-471e-9b16-98a371fa70cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:06.499341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:06.524903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6e5f5-a4cc-495d-ae38-6000e73fc31b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:06.525343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6e5f5-a4cc-495d-ae38-6000e73fc31b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:06.525852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6e5f5-a4cc-495d-ae38-6000e73fc31b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:06.528963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 359/721] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:06.539130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14607258-26ff-43c4-993f-fefebaa7066c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:06.539620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14607258-26ff-43c4-993f-fefebaa7066c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:06.540082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14607258-26ff-43c4-993f-fefebaa7066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:06.576910 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 363/722] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:44:06.589211 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-14607258-26ff-43c4-993f-fefebaa7066c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:06.591081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-14607258-26ff-43c4-993f-fefebaa7066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:06.644754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743ff44-6231-471e-9b16-98a371fa70cc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:06.769720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 360/723] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 03:44:06.843048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14607258-26ff-43c4-993f-fefebaa7066c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:06.875278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743ff44-6231-471e-9b16-98a371fa70cc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:06.947268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc82649-f718-4672-9680-a6604019e118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:06.949167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82649-f718-4672-9680-a6604019e118 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:06.950475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82649-f718-4672-9680-a6604019e118 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:06.951081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82649-f718-4672-9680-a6604019e118 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:06.976659 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc82649-f718-4672-9680-a6604019e118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-bfc82649-f718-4672-9680-a6604019e118 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:06.985818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:06.986822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:06.987720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:06.991132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 364/724] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:07.004224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc82649-f718-4672-9680-a6604019e118 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:07.089462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14607258-26ff-43c4-993f-fefebaa7066c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:07.143852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:07.149083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:07.186788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09762120-49e8-463e-b813-9de113bb485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:07.188808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09762120-49e8-463e-b813-9de113bb485b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:07.189220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09762120-49e8-463e-b813-9de113bb485b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:07.189620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09762120-49e8-463e-b813-9de113bb485b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:07.228136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09762120-49e8-463e-b813-9de113bb485b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-09762120-49e8-463e-b813-9de113bb485b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:07.246008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:07.247081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:44:07.247973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:07.248793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 361/725] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:44:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:07.257301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09762120-49e8-463e-b813-9de113bb485b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:07.517560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc82649-f718-4672-9680-a6604019e118 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:07.531874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09762120-49e8-463e-b813-9de113bb485b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:07.597863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:07.599006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:07.599541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:07.600032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:07.601575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:07.602120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:07.604877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:07.621801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 362/726] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:44:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:07.630073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:07.630427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:07.631463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:07.631987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:07.632466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:07.633452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:07.633967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:07.635599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:07.651721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 365/727] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:44:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 648 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:07.660377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:07.744632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:07.796931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643afa9-cea4-4f71-8d12-214cae76fb7c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:07.799271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:07.863050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5efc3-6d30-42ad-81a8-1fcf4cece4ba None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:07.874159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe6e74-435a-41d9-84d6-b9788a7d4226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:07.875071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6e74-435a-41d9-84d6-b9788a7d4226 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:07.875453 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6e74-435a-41d9-84d6-b9788a7d4226 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:07.876371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6e74-435a-41d9-84d6-b9788a7d4226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:07.900270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6e74-435a-41d9-84d6-b9788a7d4226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:07.901742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6e74-435a-41d9-84d6-b9788a7d4226 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:07.946133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bdfeb-7227-47aa-bd69-5ff242f86f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:07.947481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdfeb-7227-47aa-bd69-5ff242f86f55 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:07.947922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdfeb-7227-47aa-bd69-5ff242f86f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:07.948362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdfeb-7227-47aa-bd69-5ff242f86f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-a9fe6e74-435a-41d9-84d6-b9788a7d4226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:07.962475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:07.964247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:07.965971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:07.965971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:07.965971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:07.965971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:07.965971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 366/728] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:07.970018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a9b1c-b702-458f-a2e2-b88525a3f315 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:07.970018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a9b1c-b702-458f-a2e2-b88525a3f315 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:07.970018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a9b1c-b702-458f-a2e2-b88525a3f315 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:07.973993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 367/729] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:07.975472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a9b1c-b702-458f-a2e2-b88525a3f315 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:07.979250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bdfeb-7227-47aa-bd69-5ff242f86f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:07.982015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bdfeb-7227-47aa-bd69-5ff242f86f55 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7c7bdfeb-7227-47aa-bd69-5ff242f86f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:08.006715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:08.008968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:08.011957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:08.011957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:08.011957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:08.011957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:08.011957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 363/730] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:08.011957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa721-9ea7-4c11-aca2-5855d2ccb6be None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:08.011957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa721-9ea7-4c11-aca2-5855d2ccb6be None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:08.011957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa721-9ea7-4c11-aca2-5855d2ccb6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:08.046381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3f1aa721-9ea7-4c11-aca2-5855d2ccb6be None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:08.048547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3f1aa721-9ea7-4c11-aca2-5855d2ccb6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:08.149817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a9b1c-b702-458f-a2e2-b88525a3f315 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:08.212429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:08.213487 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:08.213916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:08.214336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:08.215546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:08.216021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:08.218338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:08.243261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 368/731] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:08.243261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:08.271543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:08.320871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 364/732] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:44:08.323146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9de0-cd21-4528-9f71-b31fff9a8b59 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:08.323830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9de0-cd21-4528-9f71-b31fff9a8b59 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:08.324372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9de0-cd21-4528-9f71-b31fff9a8b59 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:08.326273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 365/733] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:08.327505 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9de0-cd21-4528-9f71-b31fff9a8b59 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:08.363362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:08.416489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3633ea1-ae5f-480e-b7e4-843b2e8fab46 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:08.465185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784021fd-f1f6-4440-abfc-f993abeba82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:08.466391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021fd-f1f6-4440-abfc-f993abeba82a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:08.466864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021fd-f1f6-4440-abfc-f993abeba82a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:08.467360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021fd-f1f6-4440-abfc-f993abeba82a None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:08.467916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9de0-cd21-4528-9f71-b31fff9a8b59 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:08.468598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784021fd-f1f6-4440-abfc-f993abeba82a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:08.469100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784021fd-f1f6-4440-abfc-f993abeba82a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:08.471384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784021fd-f1f6-4440-abfc-f993abeba82a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:08.494872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 369/734] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:08.494872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30a7f-f3f4-485b-86c6-edee2c86d2b2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:08.494872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30a7f-f3f4-485b-86c6-edee2c86d2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:08.494872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30a7f-f3f4-485b-86c6-edee2c86d2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:08.517654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0cd30a7f-f3f4-485b-86c6-edee2c86d2b2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:08.519500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0cd30a7f-f3f4-485b-86c6-edee2c86d2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:08.530488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:08.531779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:08.532231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:08.532758 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:08.543442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:08.546017 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:08.563295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:08.564619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:44:08.565812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:08.569748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 366/735] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:08.572301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:08.621197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 370/736] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:44:08.629378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd30a7f-f3f4-485b-86c6-edee2c86d2b2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:08.647336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:08.699625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:08.725227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6969b3a-6da9-4dae-bdf4-09fffaf54f54 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:08.764778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4230b3d-7c31-49e3-af53-ceafcb974465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:08.765628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4230b3d-7c31-49e3-af53-ceafcb974465 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:08.766028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4230b3d-7c31-49e3-af53-ceafcb974465 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:08.766424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4230b3d-7c31-49e3-af53-ceafcb974465 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:08.767413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4230b3d-7c31-49e3-af53-ceafcb974465 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:08.768996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4230b3d-7c31-49e3-af53-ceafcb974465 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:08.774594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd30a7f-f3f4-485b-86c6-edee2c86d2b2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:08.811917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:08.812732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:08.813015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:08.813291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:08.821631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:08.823145 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:08.839033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:08.840786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 371/737] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:44:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:08.843098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:08.898765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 367/738] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:44:08 2022] POST /identity/v3/services => generated 251 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:08.898765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4230b3d-7c31-49e3-af53-ceafcb974465 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:08.944017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:09.021507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c689a-73f4-49a7-b230-2f3a1cbd3929 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:09.100377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4230b3d-7c31-49e3-af53-ceafcb974465 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:09.114033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:09.114893 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:09.115391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:09.115836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:09.117050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:09.118424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:09.120134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:09.149539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 372/739] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 1 03:44:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:09.150555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:09.169387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee254d-56c6-47e0-a94e-f73b2a5cbd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:09.170412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee254d-56c6-47e0-a94e-f73b2a5cbd8f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:09.170839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee254d-56c6-47e0-a94e-f73b2a5cbd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:09.171897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee254d-56c6-47e0-a94e-f73b2a5cbd8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:09.182120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:09.194726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee254d-56c6-47e0-a94e-f73b2a5cbd8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-abee254d-56c6-47e0-a94e-f73b2a5cbd8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:09.209961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:09.211130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:09.212112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:09.212892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 368/740] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:09.214334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee254d-56c6-47e0-a94e-f73b2a5cbd8f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:09.321244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd96477-9a50-4e90-ae40-471dcbae26ac None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:09.355398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:09.384465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:09.385543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:09.386015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:09.386487 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:09.388161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:09.388687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:09.391626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:09.432819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 373/741] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 1 03:44:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:09.432819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:09.437489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee254d-56c6-47e0-a94e-f73b2a5cbd8f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:09.509479 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bd577-15be-47bb-897a-b9900e0c67c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:09.511468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bd577-15be-47bb-897a-b9900e0c67c1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:09.512373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bd577-15be-47bb-897a-b9900e0c67c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:09.512910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bd577-15be-47bb-897a-b9900e0c67c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:09.640341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2be9e5-95aa-4ebc-beab-ce21338cb0ac None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:09.659993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:09.677450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bd577-15be-47bb-897a-b9900e0c67c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:09.680876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bd577-15be-47bb-897a-b9900e0c67c1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:09.725010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:09.726519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:09.727422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:09.728153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:09.729935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:09.730766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:09.733770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:09.749061 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 374/742] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:44:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:09.751100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:09.825906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:09.876009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 369/743] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:09 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:44:09.876941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bd577-15be-47bb-897a-b9900e0c67c1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:09.922599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e3bce-dafb-4723-b4e5-77c837195468 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:09.971605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0ce5f-3dac-4b38-8be3-8c174e48f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:09.972862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0ce5f-3dac-4b38-8be3-8c174e48f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:09.973152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0ce5f-3dac-4b38-8be3-8c174e48f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:09.973578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0ce5f-3dac-4b38-8be3-8c174e48f9b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.007791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:10.015629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bd577-15be-47bb-897a-b9900e0c67c1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:10.060529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1716f1-2a51-4e33-bde2-7499a8536fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:10.061328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1716f1-2a51-4e33-bde2-7499a8536fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.061609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1716f1-2a51-4e33-bde2-7499a8536fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.061887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1716f1-2a51-4e33-bde2-7499a8536fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.068188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0ce5f-3dac-4b38-8be3-8c174e48f9b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:10.069943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0ce5f-3dac-4b38-8be3-8c174e48f9b2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:10.075208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1716f1-2a51-4e33-bde2-7499a8536fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:10.076950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1716f1-2a51-4e33-bde2-7499a8536fd2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4e1716f1-2a51-4e33-bde2-7499a8536fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:10.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:10.098688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:10.100558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:10.100558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:10.100558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:10.100558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:10.100558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 370/744] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:10.101639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d0bef-d3b9-4198-98cd-e15c02165089 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.101928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d0bef-d3b9-4198-98cd-e15c02165089 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.102213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d0bef-d3b9-4198-98cd-e15c02165089 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.103813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 371/745] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:10.108390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d0bef-d3b9-4198-98cd-e15c02165089 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:10.129339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:10.210964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 375/746] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:09 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 03:44:10.217944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0ce5f-3dac-4b38-8be3-8c174e48f9b2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:10.236394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d0bef-d3b9-4198-98cd-e15c02165089 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:10.279953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff5763-0c77-437b-a095-8a648843c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:10.280789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff5763-0c77-437b-a095-8a648843c467 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.281072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff5763-0c77-437b-a095-8a648843c467 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.281347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff5763-0c77-437b-a095-8a648843c467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.293891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff5763-0c77-437b-a095-8a648843c467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:10.296441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff5763-0c77-437b-a095-8a648843c467 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:10.310110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5dff5763-0c77-437b-a095-8a648843c467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:10.323637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:10.325387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:10.327161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:10.327161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:10.327161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:10.327161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:10.327161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 372/747] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:10.329042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3834fa8-9bf1-442b-ac02-f3f3defc2041 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.329042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3834fa8-9bf1-442b-ac02-f3f3defc2041 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.329042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3834fa8-9bf1-442b-ac02-f3f3defc2041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.349994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d3834fa8-9bf1-442b-ac02-f3f3defc2041 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:10.351794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d3834fa8-9bf1-442b-ac02-f3f3defc2041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:10.367390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0ce5f-3dac-4b38-8be3-8c174e48f9b2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:10.409123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:10.410471 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.410652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.410873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.411736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:10.412099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:10.413553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:10.421557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:10.429046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 376/748] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:10.430250 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:10.463372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 373/749] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 03:44:10.463372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3834fa8-9bf1-442b-ac02-f3f3defc2041 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:10.567930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe393d9-3062-4979-a968-a9b6b24244b5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:10.607569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3834fa8-9bf1-442b-ac02-f3f3defc2041 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:10.636457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:10.637666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.638226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.638653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.646513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:10.652959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:10.653642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.654353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.654631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.655439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:10.655733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:10.656256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:10.665069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:10.665596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:10.683967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 374/750] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:10.683967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:10.693493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:10.694448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:10.700339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:10.700339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:10.700339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:10.700339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:10.700339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 377/751] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:10.700339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:10.759450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:10.822286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0391a-2030-4876-bdb9-092ceb7324e3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:10.859362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899927b-d493-4f16-8bb1-bbe8636b3655 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:10.864801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516301e-093e-446f-8289-3fc612bc6365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:10.865488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516301e-093e-446f-8289-3fc612bc6365 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.865764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516301e-093e-446f-8289-3fc612bc6365 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.866098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516301e-093e-446f-8289-3fc612bc6365 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.866787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516301e-093e-446f-8289-3fc612bc6365 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:10.868227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516301e-093e-446f-8289-3fc612bc6365 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:10.911457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:10.912358 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:10.912707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:10.913151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:10.925636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:10.927810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:10.943745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:10.945613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 375/752] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:44:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:10.947609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:10.982456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 378/753] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:44:10 2022] POST /identity/v3/services => generated 240 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:10.983389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516301e-093e-446f-8289-3fc612bc6365 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:11.049000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:11.145951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58806f-388a-4cf7-b4bf-34abc2232e0d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:11.198758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516301e-093e-446f-8289-3fc612bc6365 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:11.203303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:11.204112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.204392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.204666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.211146 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:11.214528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:11.216032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:11.244406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:11.245357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:11.246246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:11.246246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:11.246246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:11.246246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:11.246246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:11.246246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:11.246246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:11.246246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:11.246679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 376/754] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:11.248492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:11.261719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:11.262406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.262726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.263041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.263833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:11.264136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:11.265579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:11.279134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 379/755] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:11.279860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:11.376795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b84dc-1822-46a1-8145-3b6fafc6b429 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:11.390520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:11.401049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812eb0e-f980-46d4-90a1-7bfd0a4d0f81 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:11.439912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:11.440516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.440810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.441076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.441873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:11.442188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:11.443673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:11.445216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458795-e30a-46e5-9203-9ef0f3861bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:11.447540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458795-e30a-46e5-9203-9ef0f3861bec None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.448270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458795-e30a-46e5-9203-9ef0f3861bec None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.449001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458795-e30a-46e5-9203-9ef0f3861bec None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.451015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458795-e30a-46e5-9203-9ef0f3861bec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:11.451862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458795-e30a-46e5-9203-9ef0f3861bec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:11.455855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458795-e30a-46e5-9203-9ef0f3861bec None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:11.458359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 380/756] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:11.459209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:11.492359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 377/757] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:44:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:11.495244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e46b-bee7-4ffd-97d1-0c48d85d82a2 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.496046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e46b-bee7-4ffd-97d1-0c48d85d82a2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.496783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e46b-bee7-4ffd-97d1-0c48d85d82a2 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.499469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 378/758] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:11.500792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817e46b-bee7-4ffd-97d1-0c48d85d82a2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:11.503739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:11.588861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029fd73-1ba8-4c0e-a97c-24ae33839828 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:11.648476 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:11.649389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.649801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.650208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.662472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:11.664066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:11.674854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817e46b-bee7-4ffd-97d1-0c48d85d82a2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:11.679497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:11.680768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 381/759] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:11.683083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:11.737155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe13d5-4886-4315-9a7c-671b4ebc94a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:11.738766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe13d5-4886-4315-9a7c-671b4ebc94a3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.739288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe13d5-4886-4315-9a7c-671b4ebc94a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.739795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe13d5-4886-4315-9a7c-671b4ebc94a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.741406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe13d5-4886-4315-9a7c-671b4ebc94a3 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=1123437cabbd4aa09ca3b1a0996a8bfc) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:11.741975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe13d5-4886-4315-9a7c-671b4ebc94a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:11.744502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe13d5-4886-4315-9a7c-671b4ebc94a3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:11.766860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 379/760] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:11 2022] GET /identity/v3/role_assignments?user.id=1123437cabbd4aa09ca3b1a0996a8bfc&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:11.766860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a87c6-9279-4832-a5d1-a33323384bf2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.766860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a87c6-9279-4832-a5d1-a33323384bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.766860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a87c6-9279-4832-a5d1-a33323384bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.788298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-131a87c6-9279-4832-a5d1-a33323384bf2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:11.790501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-131a87c6-9279-4832-a5d1-a33323384bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:11.805419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:11.836108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20554d-2b3f-4ed5-86ef-4589f33da956 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:11.837823 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:11.888893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 380/761] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:11.890879 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a87c6-9279-4832-a5d1-a33323384bf2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:11.928968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:11.929957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:11.930887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:11.930887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:11.943799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:11.946115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:11.965110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:11.966231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:11.967127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:11.967127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:11.967127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:11.967127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:11.968989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 382/762] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:11.970088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:12.038788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a87c6-9279-4832-a5d1-a33323384bf2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:12.087834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:12.089375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:12.090121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:12.090854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:12.092799 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:12.093601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:12.096717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:12.115058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a118e86-632b-472b-91b4-1c4a20eaa8d0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:12.123343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 381/763] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:12.124639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:12.162925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:12.165516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:12.166662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:12.167012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:12.167382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:12.168319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:12.170430 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:12.186944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:12.187955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:12.188776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:12.193671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 383/764] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Jul 1 03:44:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:12.194680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:12.198567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:12.277138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc35506-2ffb-4552-aac9-834645cdbb48 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:12.325536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:12.326214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:12.326495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:12.326769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:12.328000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:12.328312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:12.329995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:12.335868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e5178-1ab0-417b-980c-b2fdfda9680c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:12.347783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 382/765] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:12.349598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:12.392003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:12.392696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:12.392985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:12.393267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:12.394830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=c41cca9ec5c94473b326e0940166ee75) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:12.395153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:12.396981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:12.403952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 384/766] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:12 2022] GET /identity/v3/role_assignments?user.id=c41cca9ec5c94473b326e0940166ee75&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:12.404828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:12.483858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:12.488760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:12.531523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f0ddb-6909-4e0f-9415-ecada30aeee6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:12.549232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14513e13-f0e2-445f-8990-b7dc444abeb8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:12.574908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:12.575771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:12.576059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:12.576341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:12.585164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:12.586669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:12.609460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:12.610628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:12.611753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 385/767] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:12.618562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:12.662419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:12.663099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:12.663413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:12.663785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:12.664741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:12.665038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:12.666391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:12.672151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 383/768] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 1 03:44:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:12.684346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:12.807960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3607d3-c61a-4195-ba92-5a63319df22a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:12.868371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:12.882115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:12.883206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:12.883725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:12.884210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:12.885680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:12.886233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:12.888952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:12.912978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 386/769] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:12.919896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3218-6a0c-4e18-bd75-e220c33ab16c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:12.922030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:13.019261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:13.020426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:13.020922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:13.021428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:13.022578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:13.023105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:13.024742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:13.043161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 384/770] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 1 03:44:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:13.058693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:13.060228 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61796871-2804-4dd7-8799-d5ce9624e0f3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:13.078868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:13.144471 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b3f10-e312-40eb-b16c-980c6908c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:13.146002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b3f10-e312-40eb-b16c-980c6908c894 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:13.146496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b3f10-e312-40eb-b16c-980c6908c894 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:13.146988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b3f10-e312-40eb-b16c-980c6908c894 None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/dbd31b58571b476bad12735ff7445d55/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:13.157180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b3f10-e312-40eb-b16c-980c6908c894 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=dbd31b58571b476bad12735ff7445d55, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:13.159087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b3f10-e312-40eb-b16c-980c6908c894 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:13.248278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 387/771] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:44:12 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/dbd31b58571b476bad12735ff7445d55/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:44:13.253126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fd904-4a46-4ce0-bce1-f80618364b9a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:13.320518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31659a0f-1896-4c8e-9e90-4eb36e43330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:13.321546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659a0f-1896-4c8e-9e90-4eb36e43330a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:13.322032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659a0f-1896-4c8e-9e90-4eb36e43330a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:13.322509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659a0f-1896-4c8e-9e90-4eb36e43330a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:13.324056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31659a0f-1896-4c8e-9e90-4eb36e43330a None admin] RBAC: Adding query filter params (interface=public, service_id=9bc2647654644e5da97c1dfc6770782a, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:13.324655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31659a0f-1896-4c8e-9e90-4eb36e43330a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:13.327217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31659a0f-1896-4c8e-9e90-4eb36e43330a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:13.342561 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 385/772] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 1 03:44:13 2022] GET /identity/v3/endpoints?service_id=9bc2647654644e5da97c1dfc6770782a&interface=public®ion_id=RegionOne => generated 194 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:13.349166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b3f10-e312-40eb-b16c-980c6908c894 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:13.448545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:13.525001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b3f10-e312-40eb-b16c-980c6908c894 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:13.570751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda00ee9-88aa-4703-b235-53fb7fdca5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:13.571381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00ee9-88aa-4703-b235-53fb7fdca5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:13.571662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00ee9-88aa-4703-b235-53fb7fdca5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:13.571935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00ee9-88aa-4703-b235-53fb7fdca5f3 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:13.572631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda00ee9-88aa-4703-b235-53fb7fdca5f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:13.574018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda00ee9-88aa-4703-b235-53fb7fdca5f3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:13.580360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 388/773] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:44:13 2022] GET /identity/v3/services => generated 772 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:13.639800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e1d25-0b4e-4664-8d4c-95eb6c969117 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:13.640136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e1d25-0b4e-4664-8d4c-95eb6c969117 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:13.640343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e1d25-0b4e-4664-8d4c-95eb6c969117 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:13.641781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 386/774] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:13.648550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfef88-c806-449a-9e8b-fa268ce5d243 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:13.648938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfef88-c806-449a-9e8b-fa268ce5d243 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:13.649112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfef88-c806-449a-9e8b-fa268ce5d243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:13.669315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-18cfef88-c806-449a-9e8b-fa268ce5d243 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:13.670575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-18cfef88-c806-449a-9e8b-fa268ce5d243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:13.845979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 389/775] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:13.906234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e1d25-0b4e-4664-8d4c-95eb6c969117 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:14.131073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:14.188200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e1d25-0b4e-4664-8d4c-95eb6c969117 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:14.277519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:14.278877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:14.280807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:14.281668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:14.283289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:14.286820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:14.330513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:14.332988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:14.334589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:14.339075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 387/776] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:13 2022] GET /identity/v3/services/network => generated 88 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:14.349276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfef88-c806-449a-9e8b-fa268ce5d243 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:14.455335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:14.590435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfef88-c806-449a-9e8b-fa268ce5d243 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:14.680958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7f703-34de-44de-91d2-218d858dd6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:14.682207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7f703-34de-44de-91d2-218d858dd6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:14.682731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7f703-34de-44de-91d2-218d858dd6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:14.683260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7f703-34de-44de-91d2-218d858dd6b9 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:14.684819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7f703-34de-44de-91d2-218d858dd6b9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:14.685384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7f703-34de-44de-91d2-218d858dd6b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:14.688332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7f703-34de-44de-91d2-218d858dd6b9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:14.698280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 390/777] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:14.708424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:15.006386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c2861-f879-45ff-ae79-4538f3997dc3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:15.009057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d976b4-1c90-4602-bf62-a0a03ef61dfb None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:15.009563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d976b4-1c90-4602-bf62-a0a03ef61dfb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:15.010048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d976b4-1c90-4602-bf62-a0a03ef61dfb None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:15.017139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 391/778] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:15.026099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f7d0-edb8-4bb5-8c25-63dda91e6261 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:15.026589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f7d0-edb8-4bb5-8c25-63dda91e6261 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:15.027096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f7d0-edb8-4bb5-8c25-63dda91e6261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:15.076203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:15.077977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:15.083538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-aba7f7d0-edb8-4bb5-8c25-63dda91e6261 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:15.084491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:15.085236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:15.086829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] RBAC: Adding query filter params (type=network) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:15.089294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aba7f7d0-edb8-4bb5-8c25-63dda91e6261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:15.090149 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:15.093039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:15.115686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 388/779] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:15.121443 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:15.131753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:15.261878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 392/780] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:15.332928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7f7d0-edb8-4bb5-8c25-63dda91e6261 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:15.373853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55781-cf40-40d6-8fcb-79032c11bf6b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:15.434374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:15.436153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:15.436942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:15.437752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:15.439572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] RBAC: Adding query filter params (interface=public, service_id=761eada4b52340be8cdae9d514d77c00, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:15.440392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:15.443540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:15.453956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 389/781] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 1 03:44:15 2022] GET /identity/v3/endpoints?service_id=761eada4b52340be8cdae9d514d77c00&interface=public®ion_id=RegionOne => generated 194 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:15.461762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:15.462811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:15.595906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:15.660145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d147b33-b33e-461c-93b8-5b65154a1bf9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:15.717695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7f7d0-edb8-4bb5-8c25-63dda91e6261 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:15.727215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b61068-71ed-441c-85b5-64cac34d7f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:15.728801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61068-71ed-441c-85b5-64cac34d7f77 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:15.729532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61068-71ed-441c-85b5-64cac34d7f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:15.730302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61068-71ed-441c-85b5-64cac34d7f77 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:15.731846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b61068-71ed-441c-85b5-64cac34d7f77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:15.734696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b61068-71ed-441c-85b5-64cac34d7f77 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:15.746192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 390/782] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:44:15 2022] GET /identity/v3/services => generated 772 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:15.785446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389ad5b-c975-43d0-840f-5765284a2435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:15.786281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389ad5b-c975-43d0-840f-5765284a2435 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:15.786567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389ad5b-c975-43d0-840f-5765284a2435 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:15.786843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389ad5b-c975-43d0-840f-5765284a2435 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:15.810300 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389ad5b-c975-43d0-840f-5765284a2435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b389ad5b-c975-43d0-840f-5765284a2435 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:15.822635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:15.824468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:15.826077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 393/783] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 492 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:15.833255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b61068-71ed-441c-85b5-64cac34d7f77 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:16.036285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b61068-71ed-441c-85b5-64cac34d7f77 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:16.096700 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:16.107345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497582e-38dd-42a0-8964-1bedaf019850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:16.108890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497582e-38dd-42a0-8964-1bedaf019850 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:16.109387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497582e-38dd-42a0-8964-1bedaf019850 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:16.109868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497582e-38dd-42a0-8964-1bedaf019850 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:16.111311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497582e-38dd-42a0-8964-1bedaf019850 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:16.111851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497582e-38dd-42a0-8964-1bedaf019850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:16.115011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497582e-38dd-42a0-8964-1bedaf019850 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:16.132471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 391/784] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:44:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:16.138474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389ad5b-c975-43d0-840f-5765284a2435 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:16.339753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:16.353462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389ad5b-c975-43d0-840f-5765284a2435 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:16.437962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:16.439380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:16.439881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:16.440348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:16.449434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f199-ca5a-4ce2-a310-cff85856d739 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:16.450422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f199-ca5a-4ce2-a310-cff85856d739 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:16.451196 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f199-ca5a-4ce2-a310-cff85856d739 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:16.459264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:16.459264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:16.459921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 392/785] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:16.465196 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c6ae-b335-4f23-b294-f5ee3fa619c4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:16.466094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c6ae-b335-4f23-b294-f5ee3fa619c4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:16.466856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c6ae-b335-4f23-b294-f5ee3fa619c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:16.494374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:16.497283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:16.499102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:16.499102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:16.499102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:16.499102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:16.502320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4198c6ae-b335-4f23-b294-f5ee3fa619c4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:16.503637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 394/786] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:16.504614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:16.505656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4198c6ae-b335-4f23-b294-f5ee3fa619c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:16.658420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 393/787] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:16.728985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198c6ae-b335-4f23-b294-f5ee3fa619c4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:16.761676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3666272-63eb-4e63-9cfd-d19c9e2fecfe None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:16.839477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:16.840166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:16.840450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:16.840728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:16.841477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:16.841791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:16.843306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:16.878237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 395/788] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:16.882287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:16.895859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:16.976037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198c6ae-b335-4f23-b294-f5ee3fa619c4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:17.019963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:17.042765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adba82-7fa8-4132-b47b-3c40139e5e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:17.044138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adba82-7fa8-4132-b47b-3c40139e5e4a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.044626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adba82-7fa8-4132-b47b-3c40139e5e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.045122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adba82-7fa8-4132-b47b-3c40139e5e4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.083950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adba82-7fa8-4132-b47b-3c40139e5e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-76adba82-7fa8-4132-b47b-3c40139e5e4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:17.091859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:17.093710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:44:17.095375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:17.096272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387ecc7-6a7c-4e0b-8620-91319635b8bd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:17.099563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 394/789] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:44:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:17.112427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adba82-7fa8-4132-b47b-3c40139e5e4a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:17.167336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8c825-5ffe-4b5f-a33a-738047af67c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:17.168727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8c825-5ffe-4b5f-a33a-738047af67c1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.169228 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8c825-5ffe-4b5f-a33a-738047af67c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.169729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8c825-5ffe-4b5f-a33a-738047af67c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.185767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8c825-5ffe-4b5f-a33a-738047af67c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:17.188654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8c825-5ffe-4b5f-a33a-738047af67c1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d8d8c825-5ffe-4b5f-a33a-738047af67c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:17.228306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:17.230110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 396/790] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:17.233246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163830-a89e-45f7-bf34-8805d273aebe None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.243912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163830-a89e-45f7-bf34-8805d273aebe None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.243912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163830-a89e-45f7-bf34-8805d273aebe None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.243912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 397/791] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:17.244607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe163830-a89e-45f7-bf34-8805d273aebe None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:17.266015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adba82-7fa8-4132-b47b-3c40139e5e4a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:17.337748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b406a-36c5-4480-85b5-7a294db2c69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:17.339142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b406a-36c5-4480-85b5-7a294db2c69b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.339654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b406a-36c5-4480-85b5-7a294db2c69b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.340102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b406a-36c5-4480-85b5-7a294db2c69b None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.341690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b406a-36c5-4480-85b5-7a294db2c69b None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:17.342203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b406a-36c5-4480-85b5-7a294db2c69b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:17.345078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b406a-36c5-4480-85b5-7a294db2c69b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:17.364027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 395/792] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:44:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:17.366387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389159f6-6857-4193-a72e-ba81b6cd78de None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.367109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389159f6-6857-4193-a72e-ba81b6cd78de None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.367643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389159f6-6857-4193-a72e-ba81b6cd78de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.382699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:17.405444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-389159f6-6857-4193-a72e-ba81b6cd78de None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:17.408872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-389159f6-6857-4193-a72e-ba81b6cd78de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:17.436345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe163830-a89e-45f7-bf34-8805d273aebe None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:17.470834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:17.511310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 396/793] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:17.511310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389159f6-6857-4193-a72e-ba81b6cd78de None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:17.520013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:17.521152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.521643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.527395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.529272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:17.529804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:17.536184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:17.582680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 398/794] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 1 03:44:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:17.588228 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:17.678406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389159f6-6857-4193-a72e-ba81b6cd78de None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:17.733394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:17.734183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.734464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.734733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.737055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82d74-6b92-4525-81b6-b869f0677077 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:17.745722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:17.747185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:17.766683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:17.767655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:17.768586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:17.768586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:17.768586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:17.768586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:17.769312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:17.769386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 397/795] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:17.769386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:17.786408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169be71-784f-49f6-9ad1-67445963a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:17.787561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169be71-784f-49f6-9ad1-67445963a2cc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.788082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169be71-784f-49f6-9ad1-67445963a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.788531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169be71-784f-49f6-9ad1-67445963a2cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.808768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169be71-784f-49f6-9ad1-67445963a2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:17.812968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d169be71-784f-49f6-9ad1-67445963a2cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:17.817553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:17.818702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:17.819739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:17.821847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 399/796] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:17.823400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfb316-cdda-4145-897c-aa30cb4ed691 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.823836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfb316-cdda-4145-897c-aa30cb4ed691 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.824112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfb316-cdda-4145-897c-aa30cb4ed691 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:17.825588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 400/797] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:17.826333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfb316-cdda-4145-897c-aa30cb4ed691 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:17.930692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13911a6f-4441-45ce-b017-b99bda888fe3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:17.971955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfb316-cdda-4145-897c-aa30cb4ed691 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:17.991467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b191d-00f6-4abe-8568-5f66d6ee109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:17.992276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b191d-00f6-4abe-8568-5f66d6ee109f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:17.992556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b191d-00f6-4abe-8568-5f66d6ee109f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:17.992849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b191d-00f6-4abe-8568-5f66d6ee109f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.005691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b191d-00f6-4abe-8568-5f66d6ee109f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.007146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b191d-00f6-4abe-8568-5f66d6ee109f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-553b191d-00f6-4abe-8568-5f66d6ee109f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:18.025381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:18.026480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:18.027668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:18.027668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:18.027668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:18.027668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:18.027668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 398/798] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:18.028136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b6f4-376d-4c40-9cb0-d738fe0c1754 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.028445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b6f4-376d-4c40-9cb0-d738fe0c1754 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.028587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b6f4-376d-4c40-9cb0-d738fe0c1754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.040156 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:18.040829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.041123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.041394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.042136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:18.042447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.043944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.047940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ed57b6f4-376d-4c40-9cb0-d738fe0c1754 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:18.049159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ed57b6f4-376d-4c40-9cb0-d738fe0c1754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:18.062605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 401/799] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:18.062605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:18.116227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:18.155250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 399/800] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:18.155250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57b6f4-376d-4c40-9cb0-d738fe0c1754 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:18.165557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:18.197560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48104275-b847-4695-af94-6ea9555ed4eb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:18.239175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:18.239981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.240538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.240538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.241332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:18.241640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.243565 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.252974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 402/801] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:18.254306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:18.293136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57b6f4-376d-4c40-9cb0-d738fe0c1754 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:18.336408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:18.337360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.337783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.338211 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.339082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:18.339563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.341064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.355451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 400/802] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:18.356305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:18.417584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63528827-9d5d-4b41-a341-98aeb0c73c85 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:18.427702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:18.443423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:18.464142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:18.464898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.465181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.465451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.474194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.475715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.484580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a4b7-2024-4509-8f47-5d10284b7a33 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:18.498572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:18.500992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:18.501990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 403/803] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:18.502595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:18.546340 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:18.547719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.548126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.548408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.549318 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.552001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:18.572285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:18.573424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 03:44:18.574275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:18.575064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 401/804] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:18.575940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:18.666125 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa21383-83e9-4975-9b55-9de810dec3e5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:18.725147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:18.725952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.726240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.726667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.731309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9535b-96af-4090-b465-3c416f28c6ca None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:18.738510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.740011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:18.766197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:18.767593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:18.770136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:18.770136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:18.770136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:18.770136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:18.770136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 404/805] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:18.770136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:18.782842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:18.791205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:18.792595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.793058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.793157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:18.793564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.805832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.808565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:18.828989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:18.830162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:18.832229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:18.832229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:18.832229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:18.832229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:18.832229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:18.832229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:18.832229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:18.832229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:18.836767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 402/806] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:18.837716 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:18.911040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37d14f-2253-4d1f-90f6-d46749a583fe None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:18.950938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:18.951642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:18.951928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:18.952204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:18.953055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:18.953377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:18.954818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:18.971424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67c223-250a-4be8-8330-80ac7d4c7ab1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:18.973932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 405/807] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:18.976162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.015792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:19.016677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.017042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.017506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.018302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:19.019170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:19.020597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:19.032713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 403/808] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:19.032713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.072776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:19.095637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:19.096650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe959fa-8c6e-4d0e-a01c-819587d886a0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:19.147519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d230ce4-83bc-4810-8bfe-d9ff07c88d66 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:19.149820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564a038-f5bf-4bbd-a731-c28b0d434cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:19.150602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a038-f5bf-4bbd-a731-c28b0d434cff None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.150875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a038-f5bf-4bbd-a731-c28b0d434cff None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.151148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a038-f5bf-4bbd-a731-c28b0d434cff None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.151914 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564a038-f5bf-4bbd-a731-c28b0d434cff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:19.152215 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564a038-f5bf-4bbd-a731-c28b0d434cff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:19.153609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564a038-f5bf-4bbd-a731-c28b0d434cff None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:19.165676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 406/809] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:19.166992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa902d64-1bef-4437-b93d-56d69c60ee41 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.167301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa902d64-1bef-4437-b93d-56d69c60ee41 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.167621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa902d64-1bef-4437-b93d-56d69c60ee41 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.168828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 407/810] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:19.169525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa902d64-1bef-4437-b93d-56d69c60ee41 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.194186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:19.194863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.195335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.195614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.196658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:19.196971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:19.198433 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:19.212262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 404/811] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:19.213100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.280428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa902d64-1bef-4437-b93d-56d69c60ee41 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:19.333821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:19.334544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.334817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.335078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.337581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a997178-abc3-40a0-b31b-4b2abbcb7464 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:19.338377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:19.347206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:19.348715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:19.358587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:19.367908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:19.376221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:19.378491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:19.378491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:19.378491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:19.378491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:19.378491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 408/812] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:19.378491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.395960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22161f00-79c9-425f-b567-3362fda678d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:19.396659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22161f00-79c9-425f-b567-3362fda678d7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.396941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22161f00-79c9-425f-b567-3362fda678d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.397212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22161f00-79c9-425f-b567-3362fda678d7 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.397947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22161f00-79c9-425f-b567-3362fda678d7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:19.398257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22161f00-79c9-425f-b567-3362fda678d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:19.399729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22161f00-79c9-425f-b567-3362fda678d7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:19.405395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 405/813] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 1 03:44:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:19.406775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa71c35-5964-4af9-9a53-0a97256dc694 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.407143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa71c35-5964-4af9-9a53-0a97256dc694 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.407465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa71c35-5964-4af9-9a53-0a97256dc694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.426952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4aa71c35-5964-4af9-9a53-0a97256dc694 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:19.428237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4aa71c35-5964-4af9-9a53-0a97256dc694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:19.504251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfc924-b6e4-4802-91bd-d8d2ae444b9e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:19.520199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 406/814] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:19.521050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa71c35-5964-4af9-9a53-0a97256dc694 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.557403 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:19.558673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.559141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.559515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.568557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:19.570017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:19.585886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:19.588209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 409/815] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:44:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:19.590402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.645306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa71c35-5964-4af9-9a53-0a97256dc694 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:19.653216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:19.662982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:19.695471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f340bd-9a1a-49ae-b425-c0a7a5a5fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:19.697092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f340bd-9a1a-49ae-b425-c0a7a5a5fa2e None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.697469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f340bd-9a1a-49ae-b425-c0a7a5a5fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.697831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f340bd-9a1a-49ae-b425-c0a7a5a5fa2e None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/1123437cabbd4aa09ca3b1a0996a8bfc/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.700185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f340bd-9a1a-49ae-b425-c0a7a5a5fa2e None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=1123437cabbd4aa09ca3b1a0996a8bfc, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:19.701803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f340bd-9a1a-49ae-b425-c0a7a5a5fa2e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:19.736615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20503ee8-d874-40fa-bbaa-762b0f8b33c0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:19.780741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 407/816] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:44:19 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/1123437cabbd4aa09ca3b1a0996a8bfc/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:44:19.781571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f340bd-9a1a-49ae-b425-c0a7a5a5fa2e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.802649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:19.803847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:19.804360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:19.805111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:19.806561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:19.807094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:19.809768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:19.840651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 410/817] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:19.840651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:19.992395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f340bd-9a1a-49ae-b425-c0a7a5a5fa2e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:20.004547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84f330-adc0-401b-803c-0b5c7772c1a7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:20.021231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:20.051316 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd143fb7-2245-4b40-9cbd-df632b856aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:20.052343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143fb7-2245-4b40-9cbd-df632b856aa7 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:20.052808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143fb7-2245-4b40-9cbd-df632b856aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:20.053264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143fb7-2245-4b40-9cbd-df632b856aa7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:20.054732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143fb7-2245-4b40-9cbd-df632b856aa7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:20.056925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143fb7-2245-4b40-9cbd-df632b856aa7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:20.069697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:20.070976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:20.071463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:20.071907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:20.073109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:20.075492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:20.089555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:20.091776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:20.093915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 411/818] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:19 2022] GET /identity/v3/services/network => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:20.095501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:20.187829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 408/819] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:44:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:20.189050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd143fb7-2245-4b40-9cbd-df632b856aa7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:20.258517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:20.277072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a307031-6db7-4095-bcb7-3abd79c4acc4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:20.364454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:20.367514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:20.368332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:20.369120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:20.370959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:20.371829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:20.375847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:20.417451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 412/820] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:44:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:20.418911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:20.438893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd143fb7-2245-4b40-9cbd-df632b856aa7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:20.468400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:20.516502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:20.519515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:20.520402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:20.521176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:20.537405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:20.540567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:20.572932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:20.574895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:20.577277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:20.577277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:20.577277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:20.577277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:20.577277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:20.577277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:20.577277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:20.577277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:20.582213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 409/821] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:20.588848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:20.615963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a70ea5-de7e-4d3f-a71d-fa6c27bb15db None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:20.684645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:20.686009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:20.686663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:20.687339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:20.689142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] RBAC: Adding query filter params (name=network) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:20.689991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:20.697127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:20.716289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 413/822] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:20.718056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:20.761734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:20.778794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b481a-8ce9-4107-a6b2-2a057b79175e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:20.822806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce03486-de56-405f-afbe-df4323f854a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:20.823851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03486-de56-405f-afbe-df4323f854a9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:20.824311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03486-de56-405f-afbe-df4323f854a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:20.824759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03486-de56-405f-afbe-df4323f854a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:20.913925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:20.936635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3b0fc-cfc3-4a1c-bfb9-d227e813cc33 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:20.951760 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce03486-de56-405f-afbe-df4323f854a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:20.954741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce03486-de56-405f-afbe-df4323f854a9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:21.005360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:21.006858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:21.007402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:21.008513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:21.010195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:21.010698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:21.012893 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:21.047269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 414/823] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:21.047269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:21.115982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:21.154514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 410/824] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:44:21.156348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce03486-de56-405f-afbe-df4323f854a9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:21.249044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce35306-f1c8-4094-80fa-560a941daea1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:21.323214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce03486-de56-405f-afbe-df4323f854a9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:21.334993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:21.336210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:21.336612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:21.336992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:21.337995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:21.338461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:21.340963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:21.354231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 415/825] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:21.355384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:21.380598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:21.404954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bfd8f-3c84-43ff-ac67-6bb8b4d9dfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:21.407080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfd8f-3c84-43ff-ac67-6bb8b4d9dfce None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:21.407788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfd8f-3c84-43ff-ac67-6bb8b4d9dfce None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:21.410239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfd8f-3c84-43ff-ac67-6bb8b4d9dfce None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/c41cca9ec5c94473b326e0940166ee75/roles/d31dcaade43b4d03a991cb76b4c830af` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:21.436274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bfd8f-3c84-43ff-ac67-6bb8b4d9dfce None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=c41cca9ec5c94473b326e0940166ee75, role_id=d31dcaade43b4d03a991cb76b4c830af)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:21.438343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bfd8f-3c84-43ff-ac67-6bb8b4d9dfce None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:21.522138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 411/826] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:44:21 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/c41cca9ec5c94473b326e0940166ee75/roles/d31dcaade43b4d03a991cb76b4c830af => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:44:21.522138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bfd8f-3c84-43ff-ac67-6bb8b4d9dfce None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:21.547013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e27248-42c1-45ba-97db-b44f316513d1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:21.600594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:21.601777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:21.602064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:21.602498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:21.617967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:21.619498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:21.629801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:21.645169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:21.646163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:21.647054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:21.647054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:21.647054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:21.647054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:21.647054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 416/827] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:21.653406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:21.788677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:21.788996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bfd8f-3c84-43ff-ac67-6bb8b4d9dfce None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:21.832978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5222cf-fd20-4128-87cb-49cf73df475f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:21.833712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5222cf-fd20-4128-87cb-49cf73df475f None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:21.833987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5222cf-fd20-4128-87cb-49cf73df475f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:21.834264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5222cf-fd20-4128-87cb-49cf73df475f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:21.835088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5222cf-fd20-4128-87cb-49cf73df475f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:21.836792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5222cf-fd20-4128-87cb-49cf73df475f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:21.848346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aee0a-5441-4ea1-8920-045f6f6c2623 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:21.901697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:21.902773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:21.903322 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:21.903821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:21.904976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:21.905580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:21.907385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:21.922189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 417/828] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:21.927310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 412/829] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:44:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:21.934399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:21.938284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2cb4c-f56d-4caa-9a07-217559a2268c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:21.938924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2cb4c-f56d-4caa-9a07-217559a2268c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:21.939521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2cb4c-f56d-4caa-9a07-217559a2268c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:21.941620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 413/830] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:21.948097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c5da-08e3-4d16-9280-7e38c7f2de47 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:21.948726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c5da-08e3-4d16-9280-7e38c7f2de47 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:21.950762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c5da-08e3-4d16-9280-7e38c7f2de47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:21.992582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e980c5da-08e3-4d16-9280-7e38c7f2de47 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:21.994447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e980c5da-08e3-4d16-9280-7e38c7f2de47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:22.090894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7a097-64aa-4e65-864e-27a0b3ab026b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:22.139395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 414/831] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:22.157060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:22.157989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:22.158279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:22.158559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:22.159548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:22.167546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:22.169170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:22.172173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980c5da-08e3-4d16-9280-7e38c7f2de47 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:22.193020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:22.193992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:22.194923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:22.194923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:22.194923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:22.194923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:22.194923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:22.194923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:22.194923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:22.194923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:22.197434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:22.197609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 418/832] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:22.204668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:22.296485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980c5da-08e3-4d16-9280-7e38c7f2de47 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:22.334936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243f47a-edf3-48b8-a469-3b0ba8e512a7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:22.358007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3d81b-9f98-4bd6-812d-679d2220dc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:22.358749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d81b-9f98-4bd6-812d-679d2220dc1b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:22.359033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d81b-9f98-4bd6-812d-679d2220dc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:22.359336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3d81b-9f98-4bd6-812d-679d2220dc1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:22.375273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3d81b-9f98-4bd6-812d-679d2220dc1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:22.383560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-dda3d81b-9f98-4bd6-812d-679d2220dc1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:22.383996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:22.384933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:22.385776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 415/833] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:22.387793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:22.388353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:22.388839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:22.389982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:22.390509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:22.391842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3d81b-9f98-4bd6-812d-679d2220dc1b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:22.392619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:22.410061 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 419/834] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:22.415971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:22.482610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:22.491630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:22.525426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3d81b-9f98-4bd6-812d-679d2220dc1b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:22.546953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b6b5-36ed-4b49-99e1-6a80cd1bcba3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:22.574733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:22.575888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:22.576273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:22.576792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:22.578060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:22.578657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:22.580285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:22.586355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c1acc-ae23-46ab-989f-1845cf866c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:22.587178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1acc-ae23-46ab-989f-1845cf866c3f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:22.587596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1acc-ae23-46ab-989f-1845cf866c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:22.587946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1acc-ae23-46ab-989f-1845cf866c3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:22.588862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c1acc-ae23-46ab-989f-1845cf866c3f None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=dbd31b58571b476bad12735ff7445d55) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:22.589249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c1acc-ae23-46ab-989f-1845cf866c3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:22.590777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c1acc-ae23-46ab-989f-1845cf866c3f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:22.591321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 416/835] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:22.597491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:22.601656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 420/836] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:22 2022] GET /identity/v3/role_assignments?user.id=dbd31b58571b476bad12735ff7445d55&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:22.699458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55468cf8-6a4c-44e0-8158-b6f08b53562d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:22.699659 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55468cf8-6a4c-44e0-8158-b6f08b53562d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:22.699934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55468cf8-6a4c-44e0-8158-b6f08b53562d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:22.701392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 421/837] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:22.709355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aade9-fc01-41e7-8ffa-ef5c5e4c9435 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:22.709355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aade9-fc01-41e7-8ffa-ef5c5e4c9435 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:22.709355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aade9-fc01-41e7-8ffa-ef5c5e4c9435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:22.737611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918737a0-0cb9-487d-a994-4d768e1c9231 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:22.739966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0f7aade9-fc01-41e7-8ffa-ef5c5e4c9435 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:22.742538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0f7aade9-fc01-41e7-8ffa-ef5c5e4c9435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:22.787329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:22.788150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:22.788610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:22.788814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:22.801145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:22.803872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:22.806768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:22.829734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:22.830689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:22.832190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:22.832190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:22.832190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:22.832190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:22.832870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 417/838] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:22.839872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:22.853355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 422/839] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:22.875509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aade9-fc01-41e7-8ffa-ef5c5e4c9435 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:22.931547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:22.968732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f0c79-920f-4700-a0e7-79900b2ebf08 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:23.013688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:23.014758 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.015067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aade9-fc01-41e7-8ffa-ef5c5e4c9435 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:23.015491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.015966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.016908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:23.017652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:23.020504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:23.037819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 418/840] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:23.043051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:23.067939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:23.069133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.069628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.070098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.070956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:23.072589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:23.086413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:23.087470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:23.088373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:23.091424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 423/841] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:23.097173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:23.107795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:23.177069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0cfb5-c2aa-4a10-a2e4-13a40a162886 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:23.225502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:23.226647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.226944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.227245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.233637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca747d9-a69c-44e5-8306-662c1447d267 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:23.236770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:23.237627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:23.239965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:23.257199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:23.258186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 03:44:23.259120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:23.266725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 419/842] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:23.268050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:23.276355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:23.277725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.278398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.279198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.279972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:23.280284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:23.282321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:23.287505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 424/843] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:23.296317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:23.387078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7717-6811-43ed-90a0-4ea01aa56407 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:23.403255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:23.428816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:23.430203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.430752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.431261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.432306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:23.432930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:23.435720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:23.436550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5d9a7-72af-4125-84a4-2a23667d19bd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:23.461817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 420/844] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 1 03:44:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:23.463195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:23.480446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:23.481563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.481877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.482165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.482880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:23.483211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:23.485179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:23.495347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 425/845] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:23.590704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42337ac6-93cc-4f8d-9922-25287aaf9041 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:23.631576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e1442-bcd3-497e-b601-a533fd984092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:23.632665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1442-bcd3-497e-b601-a533fd984092 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.633150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1442-bcd3-497e-b601-a533fd984092 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.633625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1442-bcd3-497e-b601-a533fd984092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.679826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:23.717205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e1442-bcd3-497e-b601-a533fd984092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:23.719057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e1442-bcd3-497e-b601-a533fd984092 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:23.827928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 421/846] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:44:23.833909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:23.869331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:23.949691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766ea62-c0e7-4a53-85b8-caa23a915b4d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:23.950330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7101be-4738-4ac0-84f3-102d55f3134d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.950330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7101be-4738-4ac0-84f3-102d55f3134d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.950470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7101be-4738-4ac0-84f3-102d55f3134d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.952120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 422/847] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:23.957966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717ca20-494d-476a-9146-cd4396fc1474 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.958260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717ca20-494d-476a-9146-cd4396fc1474 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.958530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717ca20-494d-476a-9146-cd4396fc1474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:23.976521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0717ca20-494d-476a-9146-cd4396fc1474 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:23.977740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0717ca20-494d-476a-9146-cd4396fc1474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:23.992786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:23.994080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:23.994636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:23.994919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.009774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:24.012062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:24.036517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:24.037452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:24.038355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:24.038355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:24.038355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:24.038355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:24.039729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 426/848] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:24.046172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:24.052669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:24.058897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 423/849] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:24.085283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717ca20-494d-476a-9146-cd4396fc1474 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:24.140163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:24.157436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dc029-cc7f-46d9-9bf9-372fd9287c09 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:24.199106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:24.199796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717ca20-494d-476a-9146-cd4396fc1474 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:24.199909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.200107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.200390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.201147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:24.201462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:24.203270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:24.226702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 427/850] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:24.227994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:24.249393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5eaf98-63e2-42c8-9554-ad9ccab9c77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:24.250198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaf98-63e2-42c8-9554-ad9ccab9c77e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.250484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaf98-63e2-42c8-9554-ad9ccab9c77e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.250762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaf98-63e2-42c8-9554-ad9ccab9c77e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.267377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5eaf98-63e2-42c8-9554-ad9ccab9c77e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-be5eaf98-63e2-42c8-9554-ad9ccab9c77e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:24.279679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:24.288296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:24.290082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 424/851] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:44:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:24.294887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5eaf98-63e2-42c8-9554-ad9ccab9c77e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:24.370588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa431cc-48a9-4584-b4cd-76bc2601eca2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:24.448142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:24.449557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:24.449557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.449557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.449557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:24.450106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:24.453540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:24.465844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 428/852] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:24.469572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:24.471421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:24.505711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5eaf98-63e2-42c8-9554-ad9ccab9c77e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:24.569814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:24.571568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.572356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.573128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.574825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:24.575664 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:24.578412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:24.594720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 425/853] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:44:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:24.603552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:24.608248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab74442-6884-4b99-9b2b-c0f8db7c7eba None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:24.652034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:24.653067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.653570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.654053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.655070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:24.656320 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:24.658424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:24.674265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 429/854] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:24.684362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:24.801024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:24.806555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736582a-576e-4a3b-bc28-7941a4ecc332 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:24.853271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:24.868928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb348b-28de-4541-8aff-f0109331a14b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:24.879893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89548b5-891f-444d-bbe5-788e6d69500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:24.881635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89548b5-891f-444d-bbe5-788e6d69500b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.882439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89548b5-891f-444d-bbe5-788e6d69500b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.884119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89548b5-891f-444d-bbe5-788e6d69500b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.907361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89548b5-891f-444d-bbe5-788e6d69500b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:24.909814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89548b5-891f-444d-bbe5-788e6d69500b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:24.930420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:24.931472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.931928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.932378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.933789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=1123437cabbd4aa09ca3b1a0996a8bfc) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:24.934305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:24.936795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b89548b5-891f-444d-bbe5-788e6d69500b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:24.940037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:24.941441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:24.942341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:24.942341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:24.942341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:24.942341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:24.943080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 426/855] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:24.946029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141a0bb-601d-4de9-a1d5-f6790944fb4b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.946029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141a0bb-601d-4de9-a1d5-f6790944fb4b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.946250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141a0bb-601d-4de9-a1d5-f6790944fb4b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.947491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 430/856] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:24 2022] GET /identity/v3/role_assignments?user.id=1123437cabbd4aa09ca3b1a0996a8bfc&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:24.951335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 427/857] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:24.953470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:24.954913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874967-77bd-444e-aa73-5a1d5f54d6c3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:24.955293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874967-77bd-444e-aa73-5a1d5f54d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:24.955682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874967-77bd-444e-aa73-5a1d5f54d6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:24.982906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f1874967-77bd-444e-aa73-5a1d5f54d6c3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:24.985364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f1874967-77bd-444e-aa73-5a1d5f54d6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:25.103787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 428/858] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:25.115480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfd17c-2de3-4fb7-b7b7-453014962d92 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:25.126384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874967-77bd-444e-aa73-5a1d5f54d6c3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:25.153741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:25.167376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:25.167376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.167376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.167376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:25.168053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:25.168547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:25.169961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:25.186236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 431/859] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:25.197398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:25.283860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874967-77bd-444e-aa73-5a1d5f54d6c3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:25.286671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:25.337076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:25.338092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.338475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.338857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:25.340107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:25.341940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:25.357016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:25.359079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:25.361447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 429/860] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Jul 1 03:44:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:25.366554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:25.420409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd7c25-313e-4501-bb50-21385097df09 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:25.467880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:25.469183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.469536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.469879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:25.479639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:25.481550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:25.502239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:25.504802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:25.507044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:25.509177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 432/861] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:44:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:25.510161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:25.589050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:25.606752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145511e-ddf4-4e2b-9d6e-cb1a2b51b1e6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:25.651310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:25.651991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.652275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.652555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:25.653281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:25.653595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:25.655111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:25.661014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 430/862] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:25.666462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:25.670179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7fe86-1a11-456a-b0f7-a666f136d121 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:25.709635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:25.725511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:25.726485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.726982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.727490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:25.728545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:25.729096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:25.730746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:25.753441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 433/863] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 1 03:44:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:25.767248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:25.826011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d852e-84b3-4209-93f8-88dcf001f306 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:25.869466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f09b9-d142-437e-8560-63d561ff6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:25.870698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f09b9-d142-437e-8560-63d561ff6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.870976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f09b9-d142-437e-8560-63d561ff6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.871277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f09b9-d142-437e-8560-63d561ff6ce6 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:25.872375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f09b9-d142-437e-8560-63d561ff6ce6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:25.872691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f09b9-d142-437e-8560-63d561ff6ce6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:25.874704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f09b9-d142-437e-8560-63d561ff6ce6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:25.881079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 431/864] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:25.882508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dd968-e3cc-47c0-93ee-bec45a459c41 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.882839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dd968-e3cc-47c0-93ee-bec45a459c41 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.883107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dd968-e3cc-47c0-93ee-bec45a459c41 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:25.884688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 432/865] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:25.890378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd4ac-fc54-4e3f-9319-1f12390c6570 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.890749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd4ac-fc54-4e3f-9319-1f12390c6570 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.891011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd4ac-fc54-4e3f-9319-1f12390c6570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:25.909871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d4cdd4ac-fc54-4e3f-9319-1f12390c6570 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:25.911160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d4cdd4ac-fc54-4e3f-9319-1f12390c6570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:25.912446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af718a37-27bc-4aed-8fff-8adc5bc187f5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:25.929360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:25.967845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd804906-91c2-4429-b3e1-a432d1760af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:25.970053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd804906-91c2-4429-b3e1-a432d1760af9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:25.970610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd804906-91c2-4429-b3e1-a432d1760af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:25.971971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd804906-91c2-4429-b3e1-a432d1760af9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:26.037202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 433/866] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:26.080196 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd4ac-fc54-4e3f-9319-1f12390c6570 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:26.094792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd804906-91c2-4429-b3e1-a432d1760af9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:26.096902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd804906-91c2-4429-b3e1-a432d1760af9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:26.125767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:26.229070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 434/867] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:44:26.234455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd4ac-fc54-4e3f-9319-1f12390c6570 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:26.235705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd804906-91c2-4429-b3e1-a432d1760af9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:26.250143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:26.293577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904eeb3-ce64-4108-b863-89b8563a478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:26.294962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904eeb3-ce64-4108-b863-89b8563a478d None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:26.295537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904eeb3-ce64-4108-b863-89b8563a478d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:26.296038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904eeb3-ce64-4108-b863-89b8563a478d None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:26.297111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904eeb3-ce64-4108-b863-89b8563a478d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:26.298956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904eeb3-ce64-4108-b863-89b8563a478d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:26.398242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 434/868] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:44:26 2022] POST /identity/v3/services => generated 251 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:44:26.420129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd804906-91c2-4429-b3e1-a432d1760af9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:26.490093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:26.491356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:26.491874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:26.492362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:26.514415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:26.517224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:26.549068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:26.550093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:26.550993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:26.550993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:26.550993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:26.550993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:26.552663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 435/869] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:26.560884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904eeb3-ce64-4108-b863-89b8563a478d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:26.724450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:26.749257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904eeb3-ce64-4108-b863-89b8563a478d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:26.797402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:26.798076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:26.798358 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:26.798625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:26.799422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:26.799737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:26.801187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:26.814860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 435/870] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:26.820726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:26.885011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:26.944559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d1830-634f-47fe-b491-4ce5bffc7fa6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:26.988516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:26.989713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:26.990176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:26.990628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:26.999351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:27.001069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:27.017344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:27.018359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:27.019240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:27.019240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:27.019240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:27.019240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:27.019240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:27.019240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:27.019240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:27.019240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:27.021503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 436/871] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:27.027335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:27.141637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2a6ee-5e4a-4766-95c2-9f94c3e2e18e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:27.183611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35682-c573-4584-810d-54203bd60f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:27.184546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35682-c573-4584-810d-54203bd60f2b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:27.184970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35682-c573-4584-810d-54203bd60f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:27.185410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35682-c573-4584-810d-54203bd60f2b None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:27.186381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35682-c573-4584-810d-54203bd60f2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:27.186859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35682-c573-4584-810d-54203bd60f2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:27.188548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35682-c573-4584-810d-54203bd60f2b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:27.204201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 436/872] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:27.210591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:27.334647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84937-36de-4ac7-8589-4546aa7a0cac None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:27.352550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:27.376749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89a3b2-7a63-43fc-8048-8292183c4690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:27.377704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a3b2-7a63-43fc-8048-8292183c4690 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:27.378128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a3b2-7a63-43fc-8048-8292183c4690 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:27.378570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a3b2-7a63-43fc-8048-8292183c4690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:27.379684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89a3b2-7a63-43fc-8048-8292183c4690 None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=c41cca9ec5c94473b326e0940166ee75) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:27.380140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89a3b2-7a63-43fc-8048-8292183c4690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:27.381724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89a3b2-7a63-43fc-8048-8292183c4690 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:27.389956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 437/873] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:27 2022] GET /identity/v3/role_assignments?user.id=c41cca9ec5c94473b326e0940166ee75&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:27.511419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d7dd4-e621-4705-b0ca-2cd1199cc2f6 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:27.511915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d7dd4-e621-4705-b0ca-2cd1199cc2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:27.512399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d7dd4-e621-4705-b0ca-2cd1199cc2f6 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:27.515431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 437/874] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:27.523470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235949d7-28c8-4c36-af20-55b2c852d9fd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:27.524424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235949d7-28c8-4c36-af20-55b2c852d9fd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:27.525216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235949d7-28c8-4c36-af20-55b2c852d9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:27.554022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-235949d7-28c8-4c36-af20-55b2c852d9fd None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:27.556970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-235949d7-28c8-4c36-af20-55b2c852d9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:27.647825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 438/875] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:27.689062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d7dd4-e621-4705-b0ca-2cd1199cc2f6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:27.701487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:27.811092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d7dd4-e621-4705-b0ca-2cd1199cc2f6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:27.848999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104079a-b866-4c66-9906-16a959571565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:27.849810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104079a-b866-4c66-9906-16a959571565 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:27.850091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104079a-b866-4c66-9906-16a959571565 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:27.850363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104079a-b866-4c66-9906-16a959571565 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:27.851025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104079a-b866-4c66-9906-16a959571565 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:27.852497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104079a-b866-4c66-9906-16a959571565 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1104079a-b866-4c66-9906-16a959571565 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:27.867486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:27.872057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:27.872903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 438/876] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:27.883249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235949d7-28c8-4c36-af20-55b2c852d9fd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:28.071324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235949d7-28c8-4c36-af20-55b2c852d9fd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:28.093458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:28.141531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222503c-618b-4358-b27f-ed8211462c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:28.142211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222503c-618b-4358-b27f-ed8211462c1a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:28.142491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222503c-618b-4358-b27f-ed8211462c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:28.142763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222503c-618b-4358-b27f-ed8211462c1a None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:28.144163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222503c-618b-4358-b27f-ed8211462c1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:28.144476 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222503c-618b-4358-b27f-ed8211462c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:28.145852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222503c-618b-4358-b27f-ed8211462c1a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:28.156016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 439/877] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:28.168379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104079a-b866-4c66-9906-16a959571565 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:28.297009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:28.363358 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104079a-b866-4c66-9906-16a959571565 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:28.448317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760d00d-46a6-4332-ad7d-b3d009ffa40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:28.449374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d00d-46a6-4332-ad7d-b3d009ffa40f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:28.449883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d00d-46a6-4332-ad7d-b3d009ffa40f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:28.450379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d00d-46a6-4332-ad7d-b3d009ffa40f None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:28.451341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760d00d-46a6-4332-ad7d-b3d009ffa40f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:28.451856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760d00d-46a6-4332-ad7d-b3d009ffa40f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:28.453450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760d00d-46a6-4332-ad7d-b3d009ffa40f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:28.465352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 439/878] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:28.574571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14a92-ab29-4c5a-990d-e30e645249cb None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:28.578169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14a92-ab29-4c5a-990d-e30e645249cb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:28.578169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14a92-ab29-4c5a-990d-e30e645249cb None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:28.579081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 440/879] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:28.585060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434a512-9b77-43dc-809f-ab03149aec85 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:28.585403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434a512-9b77-43dc-809f-ab03149aec85 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:28.585403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434a512-9b77-43dc-809f-ab03149aec85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:28.614823 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7434a512-9b77-43dc-809f-ab03149aec85 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:28.616524 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7434a512-9b77-43dc-809f-ab03149aec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:28.752829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 440/880] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:28.797943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd14a92-ab29-4c5a-990d-e30e645249cb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:28.852539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae3210-9ff8-4d3b-a2bf-25c668bf3f99 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:28.853992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae3210-9ff8-4d3b-a2bf-25c668bf3f99 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:28.854744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae3210-9ff8-4d3b-a2bf-25c668bf3f99 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:28.858065 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 441/881] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:28.871767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37de8-9d19-4f95-b6e8-af788c3e4631 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:28.871767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37de8-9d19-4f95-b6e8-af788c3e4631 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:28.871767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37de8-9d19-4f95-b6e8-af788c3e4631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:28.901059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b5e37de8-9d19-4f95-b6e8-af788c3e4631 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:28.903717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b5e37de8-9d19-4f95-b6e8-af788c3e4631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:28.976051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd14a92-ab29-4c5a-990d-e30e645249cb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:29.015190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a26ed-b286-4cf0-9232-63886504c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:29.015896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a26ed-b286-4cf0-9232-63886504c7ac None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:29.016182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a26ed-b286-4cf0-9232-63886504c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:29.016457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a26ed-b286-4cf0-9232-63886504c7ac None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:29.017199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a26ed-b286-4cf0-9232-63886504c7ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:29.019154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a26ed-b286-4cf0-9232-63886504c7ac None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:29.019997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:29.035359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 442/882] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:29.076883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e37de8-9d19-4f95-b6e8-af788c3e4631 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:29.099257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 441/883] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:44:28 2022] POST /identity/v3/services => generated 269 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:29.287779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e37de8-9d19-4f95-b6e8-af788c3e4631 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:29.364166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:29.366035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:29.366794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:29.367612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:29.369032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:29.371854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:29.395872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:29.397805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:29.399477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:29.404551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 443/884] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:29.414504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a26ed-b286-4cf0-9232-63886504c7ac None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:29.475512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:29.578449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a26ed-b286-4cf0-9232-63886504c7ac None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:29.619805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:29.620493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:29.620771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:29.621097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:29.621762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:29.622068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:29.623523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:29.629705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 442/885] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:29.639333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:29.754886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:29.848688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b00fe5-575a-4472-b90f-356a06ec6185 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:29.919937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:29.921023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:29.921497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:29.921975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:29.923372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:29.923913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:29.927259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:29.938332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 444/886] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:29.948171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:30.136174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:30.145459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52605af-310f-409c-87e7-c390baff3fe2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:30.218153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a3e1f-bd09-4dd7-b132-44c41879a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:30.219444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a3e1f-bd09-4dd7-b132-44c41879a915 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:30.219934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a3e1f-bd09-4dd7-b132-44c41879a915 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:30.220461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a3e1f-bd09-4dd7-b132-44c41879a915 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:30.222101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a3e1f-bd09-4dd7-b132-44c41879a915 None admin] RBAC: Adding query filter params (interface=public, service_id=826d3bdf61a549f09beb11ba46e7cb8f, region_id=RegionOne) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:30.222661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a3e1f-bd09-4dd7-b132-44c41879a915 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:30.225406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a3e1f-bd09-4dd7-b132-44c41879a915 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:30.241392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 443/887] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 1 03:44:29 2022] GET /identity/v3/endpoints?service_id=826d3bdf61a549f09beb11ba46e7cb8f&interface=public®ion_id=RegionOne => generated 194 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:30.249017 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:30.271430 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f297e6e-794e-4057-bf9b-6bde2b846a49 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:30.271894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f297e6e-794e-4057-bf9b-6bde2b846a49 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:30.272386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f297e6e-794e-4057-bf9b-6bde2b846a49 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:30.275359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 444/888] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:30.285546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c76fd4-b832-4fa2-ab63-2c9eb798d98f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:30.286013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c76fd4-b832-4fa2-ab63-2c9eb798d98f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:30.286504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c76fd4-b832-4fa2-ab63-2c9eb798d98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:30.324100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e8c76fd4-b832-4fa2-ab63-2c9eb798d98f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:30.327272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e8c76fd4-b832-4fa2-ab63-2c9eb798d98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:30.461435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6809b57-cf00-4482-be1d-ef42696a9fcc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:30.489253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 445/889] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:30.505972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:30.531109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f8276-156f-4108-8662-fa95d615909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:30.531840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f8276-156f-4108-8662-fa95d615909c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:30.532119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f8276-156f-4108-8662-fa95d615909c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:30.532354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c76fd4-b832-4fa2-ab63-2c9eb798d98f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:30.532498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f8276-156f-4108-8662-fa95d615909c None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:30.533073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f8276-156f-4108-8662-fa95d615909c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:30.534993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f8276-156f-4108-8662-fa95d615909c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:30.543089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 445/890] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:44:30 2022] GET /identity/v3/services => generated 1268 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 03:44:30.740881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e76dfa-4d1f-4c2c-8fc5-63ad01867005 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:30.741329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e76dfa-4d1f-4c2c-8fc5-63ad01867005 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:30.741782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e76dfa-4d1f-4c2c-8fc5-63ad01867005 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:30.744599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 446/891] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:30.750592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:30.753976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de512-1170-4081-9678-52384d73d4e2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:30.754766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de512-1170-4081-9678-52384d73d4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:30.754936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de512-1170-4081-9678-52384d73d4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:30.776374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c76fd4-b832-4fa2-ab63-2c9eb798d98f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:30.789765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-610de512-1170-4081-9678-52384d73d4e2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:30.791076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-610de512-1170-4081-9678-52384d73d4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:30.850707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207a52-71c7-474b-847f-c7deb45080f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:30.852255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207a52-71c7-474b-847f-c7deb45080f0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:30.852706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207a52-71c7-474b-847f-c7deb45080f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:30.853406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207a52-71c7-474b-847f-c7deb45080f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:30.880191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89207a52-71c7-474b-847f-c7deb45080f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-89207a52-71c7-474b-847f-c7deb45080f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:30.899748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:30.901586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:30.903408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:30.911277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 446/892] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 1 03:44:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:30.917924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207a52-71c7-474b-847f-c7deb45080f0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:30.931767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 447/893] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:30.973083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610de512-1170-4081-9678-52384d73d4e2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:31.133139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207a52-71c7-474b-847f-c7deb45080f0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:31.185827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610de512-1170-4081-9678-52384d73d4e2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:31.213971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:31.218395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:31.219765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:31.220344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:31.221697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:31.223665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:31.224300 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:31.228167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:31.248899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 447/894] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 1 03:44:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:31.262553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:31.271085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944cb9-4ae1-4496-acaa-5e408ddea22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:31.272290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944cb9-4ae1-4496-acaa-5e408ddea22b None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:31.272780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944cb9-4ae1-4496-acaa-5e408ddea22b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:31.273274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944cb9-4ae1-4496-acaa-5e408ddea22b None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:31.274649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57944cb9-4ae1-4496-acaa-5e408ddea22b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:31.277321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57944cb9-4ae1-4496-acaa-5e408ddea22b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:31.294512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:31.374346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 448/895] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:44:30 2022] POST /identity/v3/services => generated 241 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:44:31.464193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6ed9a-d2d0-4247-bff0-81d4c6e046c9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:31.541662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa45e08-69f1-4512-b2ad-bb0c185ac3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:31.542913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45e08-69f1-4512-b2ad-bb0c185ac3ae None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:31.543400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45e08-69f1-4512-b2ad-bb0c185ac3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:31.543839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45e08-69f1-4512-b2ad-bb0c185ac3ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:31.563723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa45e08-69f1-4512-b2ad-bb0c185ac3ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:31.566406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa45e08-69f1-4512-b2ad-bb0c185ac3ae None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-5fa45e08-69f1-4512-b2ad-bb0c185ac3ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:31.599880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:31.601587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:31.604796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:31.604796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:31.604796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:31.604796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:31.604796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 448/896] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:31.612300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944cb9-4ae1-4496-acaa-5e408ddea22b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:31.682094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5700-b5a6-4866-981e-ca95a867cefe None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:31.682588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5700-b5a6-4866-981e-ca95a867cefe None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:31.683082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5700-b5a6-4866-981e-ca95a867cefe None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:31.686492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 449/897] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:31.700860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f61c-df8b-4e3a-aa02-265514beaa55 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:31.701355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f61c-df8b-4e3a-aa02-265514beaa55 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:31.701852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f61c-df8b-4e3a-aa02-265514beaa55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:31.750402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d442f61c-df8b-4e3a-aa02-265514beaa55 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:31.751671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d442f61c-df8b-4e3a-aa02-265514beaa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:31.771128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944cb9-4ae1-4496-acaa-5e408ddea22b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:31.826012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:31.826587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:31.826871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:31.827147 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:31.828328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:31.828328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:31.829874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:31.845039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 449/898] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:31.847764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:31.855398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:31.859342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 450/899] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:31.879091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442f61c-df8b-4e3a-aa02-265514beaa55 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:31.913160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:32.007538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442f61c-df8b-4e3a-aa02-265514beaa55 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:32.041144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a27627-9f8d-4503-a2c8-65a8b2239b3e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:32.056204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:32.057401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:32.058515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:32.058797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:32.059500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:32.060886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:32.076370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:32.077376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:32.078305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:32.083255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 451/900] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Jul 1 03:44:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:32.084634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:32.109141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:32.110534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:32.111246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:32.112311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:32.126943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:32.129988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:32.150792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:32.165533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:32.167604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:32.169745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:32.179968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 450/901] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:44:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:32.183938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:32.207591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bad38-7107-4776-b418-a11908982f9c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:32.253938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:32.254835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:32.254990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:32.255300 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:32.255923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:32.256237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:32.257643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:32.267342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 452/902] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:32.270551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:32.359942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:32.376691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a8982-c033-4b32-b4dd-8c669eb64ef1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:32.401785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fafe8-4c97-4b6a-9e79-c76c9cc6f837 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:32.429066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:32.443129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:32.444894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:32.445824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:32.446551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:32.448354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:32.449141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:32.452085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:32.464062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:32.466956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:32.467536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:32.468037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:32.469541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:32.470071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:32.473862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:32.495535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 453/904] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:32.495951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 451/904] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 1 03:44:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:32.497489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:32.497882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:32.628259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d6eb-e636-4a21-8cd3-6b116ffc9b49 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:32.666442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:32.667134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:32.667433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:32.667708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:32.668503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] RBAC: Adding query filter params (interface=public, service_id=60559ef96f4d4318af3bd8c1622c026d, region_id=RegionOne) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:32.668816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:32.670195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:32.677252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 454/905] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 1 03:44:32 2022] GET /identity/v3/endpoints?service_id=60559ef96f4d4318af3bd8c1622c026d&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:32.686416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:32.692731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8669746-9d67-4c23-9463-7b1fbf0e0016 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:32.752455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de7f2a-7ff7-484c-a80c-19cb9828cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:32.753988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7f2a-7ff7-484c-a80c-19cb9828cabe None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:32.754745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7f2a-7ff7-484c-a80c-19cb9828cabe None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:32.755539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7f2a-7ff7-484c-a80c-19cb9828cabe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:32.786334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:32.794868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:32.891961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd0243-3141-464d-b5bf-b02f31ecd058 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:32.908265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de7f2a-7ff7-484c-a80c-19cb9828cabe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:32.912205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de7f2a-7ff7-484c-a80c-19cb9828cabe None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:32.956903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd010f1-08d7-485b-9ece-f4b26db0e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:32.958468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd010f1-08d7-485b-9ece-f4b26db0e8ee None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:32.959262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd010f1-08d7-485b-9ece-f4b26db0e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:32.960016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd010f1-08d7-485b-9ece-f4b26db0e8ee None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:32.961510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd010f1-08d7-485b-9ece-f4b26db0e8ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:32.964314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd010f1-08d7-485b-9ece-f4b26db0e8ee None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:32.977262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 455/906] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:44:32 2022] GET /identity/v3/services => generated 1497 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 03:44:33.110980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 452/907] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:44:33.120875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd010f1-08d7-485b-9ece-f4b26db0e8ee None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:33.315499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd010f1-08d7-485b-9ece-f4b26db0e8ee None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:33.337451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:33.382231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:33.383598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:33.384359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:33.385136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:33.406800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:33.410190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:33.411382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dba046-93de-44f3-b4d9-a95931b12906 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:33.411675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dba046-93de-44f3-b4d9-a95931b12906 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:33.411940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dba046-93de-44f3-b4d9-a95931b12906 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:33.413592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 453/908] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:33.418930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952d5e4-c870-45ff-aea4-6b183bb315e3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:33.419354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952d5e4-c870-45ff-aea4-6b183bb315e3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:33.419535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952d5e4-c870-45ff-aea4-6b183bb315e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:33.439625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6952d5e4-c870-45ff-aea4-6b183bb315e3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:33.440862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6952d5e4-c870-45ff-aea4-6b183bb315e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:33.454009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:33.455005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:33.455984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:33.455984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:33.455984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:33.455984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:33.457771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 456/909] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:33.463162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:33.499500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:33.534146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 454/910] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:33.560527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952d5e4-c870-45ff-aea4-6b183bb315e3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:33.638857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ff5ad-d365-43b3-97b0-31c2a268cf48 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:33.733846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:33.735581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:33.735856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:33.736083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:33.737386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:33.737696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:33.739100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:33.745066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952d5e4-c870-45ff-aea4-6b183bb315e3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:33.766984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 457/911] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:33.768038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:33.796795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:33.808243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a000c-bd2f-4f9a-a778-4d0513cea619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:33.809055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a000c-bd2f-4f9a-a778-4d0513cea619 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:33.809355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a000c-bd2f-4f9a-a778-4d0513cea619 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:33.813227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a000c-bd2f-4f9a-a778-4d0513cea619 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:33.813227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a000c-bd2f-4f9a-a778-4d0513cea619 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:33.813227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a000c-bd2f-4f9a-a778-4d0513cea619 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ef0a000c-bd2f-4f9a-a778-4d0513cea619 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:33.824730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:33.825808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 03:44:33.826655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:33.831466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 455/912] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:33.832954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71030f99-e83c-4572-9cf6-fbfa9ad33ce2 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:33.833238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71030f99-e83c-4572-9cf6-fbfa9ad33ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:33.833506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71030f99-e83c-4572-9cf6-fbfa9ad33ce2 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:33.835016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 456/913] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:33.835616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71030f99-e83c-4572-9cf6-fbfa9ad33ce2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:33.896680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:33.903337 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f5d8d-261f-48e4-9c8f-0b9e2175d6fc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:33.949518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfcf7a-f70f-4a73-b39c-d619ac3ec427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:33.950730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfcf7a-f70f-4a73-b39c-d619ac3ec427 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:33.951262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfcf7a-f70f-4a73-b39c-d619ac3ec427 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:33.951802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfcf7a-f70f-4a73-b39c-d619ac3ec427 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:33.964850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfcf7a-f70f-4a73-b39c-d619ac3ec427 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:33.967544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfcf7a-f70f-4a73-b39c-d619ac3ec427 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:33.972183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71030f99-e83c-4572-9cf6-fbfa9ad33ce2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e3bfcf7a-f70f-4a73-b39c-d619ac3ec427 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:33.993838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:33.994861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:33.995780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:33.995780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:33.995780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:33.995780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:33.995780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:33.995780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:33.995780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:33.995780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:33.996224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 458/914] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:33.998334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb6975-bcf7-4316-b4ea-9f0792715d49 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:33.998841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb6975-bcf7-4316-b4ea-9f0792715d49 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:33.999486 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb6975-bcf7-4316-b4ea-9f0792715d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:34.024615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:34.025380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:34.025727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:34.026236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:34.026957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:34.027377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:34.028840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:34.036108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-16eb6975-bcf7-4316-b4ea-9f0792715d49 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:34.036108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-16eb6975-bcf7-4316-b4ea-9f0792715d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:34.043088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 457/915] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:34.050206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:34.146319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 459/916] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:34.148159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb6975-bcf7-4316-b4ea-9f0792715d49 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:34.166455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:34.186128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4b525-028d-4538-a75d-fc615a6e8976 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:34.197097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:34.228814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:34.229493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:34.229765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:34.230043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:34.230841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:34.231153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:34.232725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:34.246766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 458/917] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:34.247658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:34.310901 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb6975-bcf7-4316-b4ea-9f0792715d49 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:34.358097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:34.359024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:34.359024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:34.359388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:34.360046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:34.360354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:34.362295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:34.374981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 460/918] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 1 03:44:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:34.374981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:34.424103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5173ba4-dcdc-4204-9a34-dc3db1f0ec84 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:34.477948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:34.487739 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:34.488626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:34.488883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:34.489208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:34.489927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:34.491277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:34.504349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:34.506249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:34.507981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 459/919] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:34.509129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:34.514252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e71595-1c80-4fed-8b5a-0c2c2d8bff84 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:34.552742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:34.564067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:34.568323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:34.568323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:34.568323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:34.568323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] RBAC: Adding query filter params (role.id=411d0fd9d6f2463989b3b1b51a2f2c0d, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=c41cca9ec5c94473b326e0940166ee75) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:34.568323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:34.568323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:34.579754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 461/920] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:34 2022] GET /identity/v3/role_assignments?user.id=c41cca9ec5c94473b326e0940166ee75&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=411d0fd9d6f2463989b3b1b51a2f2c0d => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:34.579754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:34.721638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf6fb8-ca35-4d8d-b2e5-2b5c3747f206 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:34.737965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb433af1-862f-44b5-8e34-5152e870ced4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:34.788036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:34.788837 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:34.789461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:34.789813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:34.790626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:34.791004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:34.792840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:34.800427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 462/921] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:34.807075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:34.811079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:34.816323 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63471-5efb-461a-a58e-6b4c2d190e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:34.817695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63471-5efb-461a-a58e-6b4c2d190e46 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:34.818333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63471-5efb-461a-a58e-6b4c2d190e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:34.818961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63471-5efb-461a-a58e-6b4c2d190e46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:34.821703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb63471-5efb-461a-a58e-6b4c2d190e46 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:34.823657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb63471-5efb-461a-a58e-6b4c2d190e46 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:34.898898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 460/922] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:44:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:34.970108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fac903-53e5-4487-985e-75e451204126 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:35.027535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:35.029174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:35.029756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:35.030209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:35.031842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:35.032567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:35.035604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:35.051438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 463/923] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:35.055448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63471-5efb-461a-a58e-6b4c2d190e46 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:35.187611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:35.204127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63471-5efb-461a-a58e-6b4c2d190e46 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:35.261768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a422379-39d3-421c-a67f-690cb3fc5ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:35.263342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a422379-39d3-421c-a67f-690cb3fc5ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:35.264067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a422379-39d3-421c-a67f-690cb3fc5ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:35.264762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a422379-39d3-421c-a67f-690cb3fc5ca3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:35.266513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a422379-39d3-421c-a67f-690cb3fc5ca3 None admin] RBAC: Adding query filter params (interface=public, service_id=68e57c344b1c4add8ab0d192d05b9fd5, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:35.267393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a422379-39d3-421c-a67f-690cb3fc5ca3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:35.270147 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a422379-39d3-421c-a67f-690cb3fc5ca3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:35.283625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 461/924] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 1 03:44:35 2022] GET /identity/v3/endpoints?service_id=68e57c344b1c4add8ab0d192d05b9fd5&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:35.289643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:35.403434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:35.458290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd7c3a-b4ec-4b20-a539-8ac06468e078 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:35.497127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa377c4-6b0b-45e7-94d7-d174dac0a55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:35.497804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa377c4-6b0b-45e7-94d7-d174dac0a55f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:35.498082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa377c4-6b0b-45e7-94d7-d174dac0a55f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:35.498363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa377c4-6b0b-45e7-94d7-d174dac0a55f None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:35.499063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa377c4-6b0b-45e7-94d7-d174dac0a55f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:35.500506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa377c4-6b0b-45e7-94d7-d174dac0a55f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:35.508495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 464/925] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:44:35 2022] GET /identity/v3/services => generated 1497 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 03:44:35.582299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91558d-ba13-432b-8b8e-3d47ef7f889d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:35.583301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91558d-ba13-432b-8b8e-3d47ef7f889d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:35.584096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91558d-ba13-432b-8b8e-3d47ef7f889d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:35.587618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 462/926] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:35.596652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d557bd0-3df7-40d7-9bd9-78e26244b936 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:35.597143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d557bd0-3df7-40d7-9bd9-78e26244b936 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:35.597617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d557bd0-3df7-40d7-9bd9-78e26244b936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:35.625934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2d557bd0-3df7-40d7-9bd9-78e26244b936 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:35.628192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2d557bd0-3df7-40d7-9bd9-78e26244b936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:35.720360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 465/927] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:35.759600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91558d-ba13-432b-8b8e-3d47ef7f889d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:35.933594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91558d-ba13-432b-8b8e-3d47ef7f889d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:36.001873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:36.002922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:36.003432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:36.003919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:36.005121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:36.007732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:36.032456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:36.034057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 03:44:36.034888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:36.036995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 463/928] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Jul 1 03:44:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:36.042688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d557bd0-3df7-40d7-9bd9-78e26244b936 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:36.151787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:36.226520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d557bd0-3df7-40d7-9bd9-78e26244b936 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:36.290247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:36.291478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:36.291970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:36.292455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:36.293735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:36.294291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:36.297060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:36.307121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 466/929] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:36.316679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:36.363958 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:36.499369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdef1c8-2cf3-4c1a-83c9-de163fef7d91 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:36.563105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f51db-d3b4-4360-880a-957fd82e7269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:36.564243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f51db-d3b4-4360-880a-957fd82e7269 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:36.564777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f51db-d3b4-4360-880a-957fd82e7269 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:36.565259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f51db-d3b4-4360-880a-957fd82e7269 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:36.566566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f51db-d3b4-4360-880a-957fd82e7269 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:36.567108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f51db-d3b4-4360-880a-957fd82e7269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:36.569691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f51db-d3b4-4360-880a-957fd82e7269 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:36.579675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 464/930] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:36.588791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:36.792412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4776ee0-ae49-4e0e-b05c-e74561edbd5e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:36.835967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:36.855858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962e2af-506c-45ab-af88-3f8e894ac667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:36.856950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962e2af-506c-45ab-af88-3f8e894ac667 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:36.857437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962e2af-506c-45ab-af88-3f8e894ac667 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:36.859345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962e2af-506c-45ab-af88-3f8e894ac667 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:36.859345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962e2af-506c-45ab-af88-3f8e894ac667 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:36.861799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962e2af-506c-45ab-af88-3f8e894ac667 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:37.018986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 467/931] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:44:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:37.135687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8b65-4ca7-4a14-bf06-9dbafd8922b1 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:37.135964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8b65-4ca7-4a14-bf06-9dbafd8922b1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:37.136111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8b65-4ca7-4a14-bf06-9dbafd8922b1 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:37.137914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 465/932] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:37.144643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b081-722d-40a4-a35f-b69f155f0cfb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:37.144939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b081-722d-40a4-a35f-b69f155f0cfb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:37.145296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b081-722d-40a4-a35f-b69f155f0cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:37.164604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b1d7b081-722d-40a4-a35f-b69f155f0cfb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:37.165849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b1d7b081-722d-40a4-a35f-b69f155f0cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:37.290707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 468/933] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:37.314307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab8b65-4ca7-4a14-bf06-9dbafd8922b1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:37.414278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:37.436580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab8b65-4ca7-4a14-bf06-9dbafd8922b1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:37.483648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899816c7-0c97-4b46-8152-46bb49912716 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:37.484096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899816c7-0c97-4b46-8152-46bb49912716 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:37.484557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899816c7-0c97-4b46-8152-46bb49912716 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:37.491742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 469/934] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:37.496124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc014-7df3-4028-b7fc-606db436e10f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:37.496618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc014-7df3-4028-b7fc-606db436e10f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:37.497107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc014-7df3-4028-b7fc-606db436e10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:37.504739 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75448db-05ad-41b7-acfd-6eb694535d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:37.506590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75448db-05ad-41b7-acfd-6eb694535d21 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:37.507386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75448db-05ad-41b7-acfd-6eb694535d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:37.508072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75448db-05ad-41b7-acfd-6eb694535d21 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:37.526008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0d3fc014-7df3-4028-b7fc-606db436e10f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:37.528226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0d3fc014-7df3-4028-b7fc-606db436e10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:37.529864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75448db-05ad-41b7-acfd-6eb694535d21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-e75448db-05ad-41b7-acfd-6eb694535d21 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:37.547181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:37.549090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:37.550706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 466/935] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 1 03:44:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:37.563363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75448db-05ad-41b7-acfd-6eb694535d21 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:37.697772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 470/936] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:37.698798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75448db-05ad-41b7-acfd-6eb694535d21 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:37.739817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fc014-7df3-4028-b7fc-606db436e10f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:37.755671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:37.757311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:37.758123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:37.758949 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:37.760627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:37.761546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:37.764392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:37.781694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 467/937] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 1 03:44:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:37.789288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:37.789713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:37.794685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:37.909381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fc014-7df3-4028-b7fc-606db436e10f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:37.922069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedacd69-e8e6-4da6-80ce-0650525002c3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:37.953713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:37.954754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:37.955056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:37.955853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:37.956576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:37.958028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:37.960078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING keystone.server.flask.common [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 01 03:44:37.984268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:37.985803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:37.986475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:37.987079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:38.006758 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:38.008497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.sql.core [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 03:44:38.008497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 01 03:44:38.008497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [parameters: {'id': 'd6828215936d46e89162d11cdf49a34f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 01 03:44:38.008497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 03:44:38.009641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6828215936d46e89162d11cdf49a34f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6828215936d46e89162d11cdf49a34f. Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 03:44:38.018538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 01 03:44:38.019436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application yield resource Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise self.value Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application session.commit() Jul 01 03:44:38.020388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.flush() Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application compat.raise_( Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 03:44:38.021309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 03:44:38.022241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 03:44:38.023054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [parameters: {'id': 'd6828215936d46e89162d11cdf49a34f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:38.026982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:38.028979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6828215936d46e89162d11cdf49a34f. Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:38.031614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 471/938] 10.222.0.20 () {60 vars in 1140 bytes} [Fri Jul 1 03:44:37 2022] POST /identity/v3/projects => generated 262 bytes in 291 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:38.037431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:38.051375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:38.053129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:38.056418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:38.056418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:38.056418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:38.056418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:38.060287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 468/939] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:38.066674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:38.173576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35cca9-4869-46bd-8643-7af2fe945e4f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:38.178839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:38.219997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:38.238824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dcefa-ecc4-4133-8f6b-6c4988e5cc88 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:38.247106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:38.248594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:38.249288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:38.250006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:38.251474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:38.252715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:38.256256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:38.282713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 469/940] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:38.288978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:38.310027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:38.311357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:38.311836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:38.312310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:38.327757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:38.330699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:38.362029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:38.363802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:38.365512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:38.365512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:38.365512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:38.365512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:38.365512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:38.365512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:38.365512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:38.365512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:38.371280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 472/941] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:38.374409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:38.494891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434727b7-fce9-430e-b94a-dc3f2d8f4bb6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:38.524032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:38.566455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:38.569254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:38.570079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:38.570843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:38.584264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41610a38-eade-4626-851d-57cbbda89823 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:38.586027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:38.588888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:38.617835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:38.619745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:38.621572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:38.626241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 470/942] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:44:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:38.635476 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:38.659124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:38.660962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:38.661327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:38.662007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:38.662840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:38.663437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:38.665747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:38.695326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 473/943] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:38 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:38.822678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36513101-0a38-4121-a44f-a4ce05f42ae5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:38.876415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cae80-fca7-4291-8157-6e26ff02ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:38.877641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cae80-fca7-4291-8157-6e26ff02ee65 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:38.878192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cae80-fca7-4291-8157-6e26ff02ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:38.878684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cae80-fca7-4291-8157-6e26ff02ee65 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:38.879893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cae80-fca7-4291-8157-6e26ff02ee65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:38.880415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cae80-fca7-4291-8157-6e26ff02ee65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:38.882757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cae80-fca7-4291-8157-6e26ff02ee65 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:38.913461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 471/944] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 1 03:44:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:38.920231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:38.941447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:39.036787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed3c0d-7fc3-421a-a2ca-f8b0cca9649b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.037486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed3c0d-7fc3-421a-a2ca-f8b0cca9649b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.037486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed3c0d-7fc3-421a-a2ca-f8b0cca9649b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.040322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 472/945] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:39.043779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f5a91-9023-4cc0-8ea1-e12637853bff None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:39.045143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2597794-0230-482c-b0ab-59f9be33fc58 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.045502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2597794-0230-482c-b0ab-59f9be33fc58 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.045872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2597794-0230-482c-b0ab-59f9be33fc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.063727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a2597794-0230-482c-b0ab-59f9be33fc58 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:39.065088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a2597794-0230-482c-b0ab-59f9be33fc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:39.089630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:39.092399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.092786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.092998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.101070 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:39.110011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:39.111868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:39.138442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:39.139556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:39.140452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:39.140452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:39.140452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:39.140452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:39.147437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 474/946] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:39.150814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:39.156203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 473/947] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:39.182494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2597794-0230-482c-b0ab-59f9be33fc58 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:39.237215 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:39.284660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9def94d-cbe6-4cac-a113-2d4fe5d9ccc3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:39.335679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:39.336520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.336807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.337084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.338202 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:39.338529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:39.340344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:39.349126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2597794-0230-482c-b0ab-59f9be33fc58 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:39.355909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 475/948] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:39.373285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:39.422591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:39.423439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.423769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.424080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.425036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:39.428209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:39.450042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:39.451320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:39.452682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:39.459908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 474/949] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:39.462874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:39.515417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:39.572583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993c128-2ff1-46db-83df-638cc8ba6e4c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:39.616043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:39.635482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b2579-11fc-42b5-b919-e09f3c2d024c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:39.636157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e9f91-95e8-416d-a005-402f451333db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:39.637501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e9f91-95e8-416d-a005-402f451333db None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.638048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e9f91-95e8-416d-a005-402f451333db None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.638483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e9f91-95e8-416d-a005-402f451333db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.652046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e9f91-95e8-416d-a005-402f451333db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:39.654807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e9f91-95e8-416d-a005-402f451333db None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-436e9f91-95e8-416d-a005-402f451333db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:39.675140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:39.677043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:39.678836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 476/950] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:39.681426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194579-8a5e-49d9-8b75-0f7eee657c1c None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.681724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194579-8a5e-49d9-8b75-0f7eee657c1c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.682177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194579-8a5e-49d9-8b75-0f7eee657c1c None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.684314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 477/951] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:39.695113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce194579-8a5e-49d9-8b75-0f7eee657c1c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:39.700013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d6155-8fca-4e40-9e2f-710bd1eaedb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:39.700291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d6155-8fca-4e40-9e2f-710bd1eaedb8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.701042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d6155-8fca-4e40-9e2f-710bd1eaedb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.701321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d6155-8fca-4e40-9e2f-710bd1eaedb8 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.703094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823d6155-8fca-4e40-9e2f-710bd1eaedb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:39.708466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823d6155-8fca-4e40-9e2f-710bd1eaedb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:39.708466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823d6155-8fca-4e40-9e2f-710bd1eaedb8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:39.715105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 475/952] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:39.720966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93a305-121e-4f8e-8a7f-2468b75b07f4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.721329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93a305-121e-4f8e-8a7f-2468b75b07f4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.721533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93a305-121e-4f8e-8a7f-2468b75b07f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.740868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ee93a305-121e-4f8e-8a7f-2468b75b07f4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:39.742089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ee93a305-121e-4f8e-8a7f-2468b75b07f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:39.834326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce194579-8a5e-49d9-8b75-0f7eee657c1c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:39.847291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 476/953] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:39.847291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93a305-121e-4f8e-8a7f-2468b75b07f4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:39.883724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:39.896068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:39.897474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:39.898206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:39.898704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:39.900315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:39.900879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:39.903525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:39.928797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 478/954] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:39.931934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:40.016235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:40.035703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93a305-121e-4f8e-8a7f-2468b75b07f4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:40.088461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a0457-a98c-4125-a738-5cf3d3fbd1b8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:40.103880 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:40.105119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:40.105749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:40.106211 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:40.107629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:40.108159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:40.110949 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:40.128101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 477/955] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:40.128101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:40.140565 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:40.141277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:40.141563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:40.141828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:40.142538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:40.144078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:40.157417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:40.159852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:40.160727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 479/956] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:40.162057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:40.270375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:40.297660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c187a-ef6a-44d5-bd08-0bd3ee91c99e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:40.306008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b10b4-1c62-4886-b321-5192e5328434 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:40.315110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:40.346461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1438416-13ff-4b44-9c1f-cd7f2eeebe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:40.349388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1438416-13ff-4b44-9c1f-cd7f2eeebe7c None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:40.350051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1438416-13ff-4b44-9c1f-cd7f2eeebe7c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:40.350508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1438416-13ff-4b44-9c1f-cd7f2eeebe7c None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/c41cca9ec5c94473b326e0940166ee75/roles/411d0fd9d6f2463989b3b1b51a2f2c0d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:40.353993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1438416-13ff-4b44-9c1f-cd7f2eeebe7c None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=c41cca9ec5c94473b326e0940166ee75, role_id=411d0fd9d6f2463989b3b1b51a2f2c0d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:40.354536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06d581-335a-4cd6-96b4-026a10662426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:40.355830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06d581-335a-4cd6-96b4-026a10662426 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:40.356532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06d581-335a-4cd6-96b4-026a10662426 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:40.356694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1438416-13ff-4b44-9c1f-cd7f2eeebe7c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:40.357395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06d581-335a-4cd6-96b4-026a10662426 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:40.358903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06d581-335a-4cd6-96b4-026a10662426 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:40.362096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06d581-335a-4cd6-96b4-026a10662426 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:40.467082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 478/957] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:44:40 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/c41cca9ec5c94473b326e0940166ee75/roles/411d0fd9d6f2463989b3b1b51a2f2c0d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:44:40.467885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1438416-13ff-4b44-9c1f-cd7f2eeebe7c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:40.515307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 480/958] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:44:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:40.632576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1438416-13ff-4b44-9c1f-cd7f2eeebe7c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:40.720003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5a72e-9230-4267-88d5-707a12f62ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:40.721647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5a72e-9230-4267-88d5-707a12f62ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:40.722135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5a72e-9230-4267-88d5-707a12f62ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:40.722604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5a72e-9230-4267-88d5-707a12f62ac7 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:40.723666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5a72e-9230-4267-88d5-707a12f62ac7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:40.724210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5a72e-9230-4267-88d5-707a12f62ac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:40.726783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5a72e-9230-4267-88d5-707a12f62ac7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:40.736874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 479/959] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:40.742980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06d581-335a-4cd6-96b4-026a10662426 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:40.931449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06d581-335a-4cd6-96b4-026a10662426 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:40.996397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:40.997933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:40.998699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:40.999478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:41.001035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:41.002434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:41.005409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:41.012762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f804b2-a4e6-42a3-be72-4f9a39367c53 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:41.013054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f804b2-a4e6-42a3-be72-4f9a39367c53 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:41.013302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f804b2-a4e6-42a3-be72-4f9a39367c53 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:41.014998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 480/960] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:41.015431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 481/961] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:41.021090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e8b78-3200-44a1-a4f4-8f1af55a926e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:41.021427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e8b78-3200-44a1-a4f4-8f1af55a926e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:41.021635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e8b78-3200-44a1-a4f4-8f1af55a926e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:41.033868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:41.049512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-349e8b78-3200-44a1-a4f4-8f1af55a926e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:41.052073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-349e8b78-3200-44a1-a4f4-8f1af55a926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:41.192752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 481/962] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:41.218260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:41.270095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:41.398890 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b446cbd-6d55-4b88-8513-f5c81c131632 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:41.468395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:41.469850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:41.470543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:41.471161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:41.472654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:41.475533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:41.529905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.sql.core [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 03:44:41.529905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 01 03:44:41.529905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [parameters: {'user_id': 'c5ae63085e824f8da89e1f6e50842d9d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 01 03:44:41.529905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 01 03:44:41.546661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 03:44:41.549276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application yield resource Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise self.value Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application session.commit() Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 03:44:41.551424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.flush() Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application compat.raise_( Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 03:44:41.553428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 03:44:41.555794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 03:44:41.557701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c5ae63085e824f8da89e1f6e50842d9d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:41.562973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 01 03:44:41.565845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 01 03:44:41.580787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 01 03:44:41.580787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 01 03:44:41.580787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 03:44:41.580787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 03:44:41.580787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 01 03:44:41.580787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:41.580787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 482/963] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 1 03:44:41 2022] POST /identity/v3/users => generated 160 bytes in 345 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:41.582065 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e8b78-3200-44a1-a4f4-8f1af55a926e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:41.748240 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e8b78-3200-44a1-a4f4-8f1af55a926e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:41.811867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead14a2d-63ab-4856-8def-4bb4c0e1970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:41.813210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14a2d-63ab-4856-8def-4bb4c0e1970a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:41.813698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14a2d-63ab-4856-8def-4bb4c0e1970a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:41.814203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14a2d-63ab-4856-8def-4bb4c0e1970a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:41.818078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:41.829055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14a2d-63ab-4856-8def-4bb4c0e1970a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:41.831677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14a2d-63ab-4856-8def-4bb4c0e1970a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ead14a2d-63ab-4856-8def-4bb4c0e1970a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:41.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:41.864322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:41.866442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 482/964] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:44:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:41.871734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:42.037880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:42.067906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7295-c8a1-4bd0-9c9d-6a18afd6848e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:42.137664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fc38a-53f7-402b-bd2b-321319e745bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:42.138770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc38a-53f7-402b-bd2b-321319e745bd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:42.139211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc38a-53f7-402b-bd2b-321319e745bd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:42.139700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc38a-53f7-402b-bd2b-321319e745bd None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:42.141167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fc38a-53f7-402b-bd2b-321319e745bd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:42.141710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fc38a-53f7-402b-bd2b-321319e745bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:42.145742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fc38a-53f7-402b-bd2b-321319e745bd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:42.224320 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 483/965] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 1 03:44:41 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:43.096519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab8432-fdb2-4fff-840c-8bdc4787eca7 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.097327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab8432-fdb2-4fff-840c-8bdc4787eca7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.097327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab8432-fdb2-4fff-840c-8bdc4787eca7 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:43.098825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 483/966] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:43.105591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782510-d72a-46de-a08b-6b3ee12ba92b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.106123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782510-d72a-46de-a08b-6b3ee12ba92b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.106703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8782510-d72a-46de-a08b-6b3ee12ba92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:43.129585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a8782510-d72a-46de-a08b-6b3ee12ba92b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:43.131150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a8782510-d72a-46de-a08b-6b3ee12ba92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:43.241840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 484/967] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:43.280380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab8432-fdb2-4fff-840c-8bdc4787eca7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:43.467661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab8432-fdb2-4fff-840c-8bdc4787eca7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:43.518914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95e3dc-be9a-491f-bb63-bce90d3175f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:43.519770 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e3dc-be9a-491f-bb63-bce90d3175f5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.520050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e3dc-be9a-491f-bb63-bce90d3175f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.520324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e3dc-be9a-491f-bb63-bce90d3175f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:43.532602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95e3dc-be9a-491f-bb63-bce90d3175f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-da95e3dc-be9a-491f-bb63-bce90d3175f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:43.541063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:43.542626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:43.544044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 484/968] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 1 03:44:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:43.550458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8782510-d72a-46de-a08b-6b3ee12ba92b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:43.617113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31809fee-8634-4d20-8069-db7fcd046f5f None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.617374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31809fee-8634-4d20-8069-db7fcd046f5f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.617658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31809fee-8634-4d20-8069-db7fcd046f5f None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:43.619046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 485/969] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:43.624382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e8be3-97ae-4da1-8ba1-c02dce80f13a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.624670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e8be3-97ae-4da1-8ba1-c02dce80f13a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.624956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e8be3-97ae-4da1-8ba1-c02dce80f13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:43.642709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ed8e8be3-97ae-4da1-8ba1-c02dce80f13a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:43.643962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ed8e8be3-97ae-4da1-8ba1-c02dce80f13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:43.682645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:43.715699 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:43.735358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8782510-d72a-46de-a08b-6b3ee12ba92b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:43.741693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 486/970] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:43.769786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e8be3-97ae-4da1-8ba1-c02dce80f13a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:43.784161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:43.785182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.785690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.786198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:43.787639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:43.788274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:43.790682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:43.810987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 485/971] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 1 03:44:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:43.812812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:43.899163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e8be3-97ae-4da1-8ba1-c02dce80f13a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:43.937007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4a890-4812-465e-a45d-74c5968075cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:43.937824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4a890-4812-465e-a45d-74c5968075cd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.938102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4a890-4812-465e-a45d-74c5968075cd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.938388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4a890-4812-465e-a45d-74c5968075cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:43.939053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4a890-4812-465e-a45d-74c5968075cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:43.940498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4a890-4812-465e-a45d-74c5968075cd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:43.950533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679355-1fcc-46c7-b7c0-a971d1648503 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-e7b4a890-4812-465e-a45d-74c5968075cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:43.952850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:43.961982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:43.963694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 487/972] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:43.965064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86259-6117-4691-b008-c7ef0f499730 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.965064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86259-6117-4691-b008-c7ef0f499730 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.965064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86259-6117-4691-b008-c7ef0f499730 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:43.965064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 488/973] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:43.965064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86259-6117-4691-b008-c7ef0f499730 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:43.994187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35fb62-4660-41a5-af1e-eea023e07a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:43.994900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35fb62-4660-41a5-af1e-eea023e07a8f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:43.995174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35fb62-4660-41a5-af1e-eea023e07a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:43.995486 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35fb62-4660-41a5-af1e-eea023e07a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.003538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:44.007512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35fb62-4660-41a5-af1e-eea023e07a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.009069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35fb62-4660-41a5-af1e-eea023e07a8f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.025600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-af35fb62-4660-41a5-af1e-eea023e07a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:44.041077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:44.043037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:44.049365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:44.049365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:44.049365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:44.049365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:44.049365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 486/974] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:44.049365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750360f-f8a7-499c-88f1-b8244e57567d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.049365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750360f-f8a7-499c-88f1-b8244e57567d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.049365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750360f-f8a7-499c-88f1-b8244e57567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.065836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8750360f-f8a7-499c-88f1-b8244e57567d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:44.067152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8750360f-f8a7-499c-88f1-b8244e57567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:44.087812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86259-6117-4691-b008-c7ef0f499730 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:44.127246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:44.128224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.128224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.128542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.129172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:44.129492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.130878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.136967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 489/975] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:44.137377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:44.147023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 487/976] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:44.148574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750360f-f8a7-499c-88f1-b8244e57567d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:44.256652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44a13c-c9c3-4dde-907d-b2bb74a3eb60 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:44.260532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750360f-f8a7-499c-88f1-b8244e57567d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:44.273263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:44.288271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:44.298494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:44.299158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.299466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.299744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.300463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:44.300766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.302260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.307744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:44.308965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.309445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.309931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.311291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:44.311812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.313882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.319685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 490/977] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:44.320800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:44.328921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 488/978] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:44.328921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:44.432918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1bbc0-aa78-4547-927d-62e698ca07da None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:44.448919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6ff6c-a159-43f9-80ab-ac2330dd507c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:44.483748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae3b0-d0cc-4e41-95de-a8172b5b094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:44.484663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae3b0-d0cc-4e41-95de-a8172b5b094b None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.484774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:44.485070 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae3b0-d0cc-4e41-95de-a8172b5b094b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.485389 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae3b0-d0cc-4e41-95de-a8172b5b094b None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.485492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.485753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.486056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.486299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ae3b0-d0cc-4e41-95de-a8172b5b094b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.488057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ae3b0-d0cc-4e41-95de-a8172b5b094b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.494772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.496259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:44.512627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:44.513803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:44.514816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 489/979] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:44:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:44.515547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:44.569201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:44.582078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 491/980] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:44:44 2022] POST /identity/v3/services => generated 242 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:44.582722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae3b0-d0cc-4e41-95de-a8172b5b094b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:44.650491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:44.651877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e89dd-57ef-4b79-98cc-dd7960d754d9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:44.729602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:44.731406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.731906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.732406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.733960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] RBAC: Adding query filter params (interface=admin, service_id=68e57c344b1c4add8ab0d192d05b9fd5, region_id=RegionOne) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:44.734520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.737307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.746435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 490/981] 10.222.0.20 () {56 vars in 1249 bytes} [Fri Jul 1 03:44:44 2022] GET /identity/v3/endpoints?service_id=68e57c344b1c4add8ab0d192d05b9fd5&interface=admin®ion_id=RegionOne => generated 193 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:44.752427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:44.759515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae3b0-d0cc-4e41-95de-a8172b5b094b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:44.823500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:44.824089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.824371 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.824953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.825815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:44.826300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.828164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.850288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 492/982] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 1 03:44:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:44.856793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:44.898925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8819c6f-9839-4911-89cb-2b32d6e5f8e7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:44.904494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:44.957803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eeef9-6876-4e8a-a0a1-90a5f66184b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:44.958972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eeef9-6876-4e8a-a0a1-90a5f66184b1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:44.959515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eeef9-6876-4e8a-a0a1-90a5f66184b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:44.960018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eeef9-6876-4e8a-a0a1-90a5f66184b1 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:44.961275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eeef9-6876-4e8a-a0a1-90a5f66184b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:44.963919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eeef9-6876-4e8a-a0a1-90a5f66184b1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:44.967927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:44.976020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 491/983] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:44:44 2022] GET /identity/v3/services => generated 1727 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 03:44:44.981610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b2afd-42fd-4649-8706-a272ec3d7d96 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:45.026649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72696b02-f422-4b66-b1e5-6544371da23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:45.027380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72696b02-f422-4b66-b1e5-6544371da23d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:45.027677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72696b02-f422-4b66-b1e5-6544371da23d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:45.027956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72696b02-f422-4b66-b1e5-6544371da23d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:45.114313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72696b02-f422-4b66-b1e5-6544371da23d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:45.115832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72696b02-f422-4b66-b1e5-6544371da23d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:45.221859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:45.233192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 493/984] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:44 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 03:44:45.238924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eeef9-6876-4e8a-a0a1-90a5f66184b1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:45.348700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eeef9-6876-4e8a-a0a1-90a5f66184b1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:45.386043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:45.386764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:45.387050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:45.387357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:45.399926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:45.401396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:45.424641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:45.426401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:45.428204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:45.428204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:45.428204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:45.428204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:45.431354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 492/985] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:45.433691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72696b02-f422-4b66-b1e5-6544371da23d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:45.573469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72696b02-f422-4b66-b1e5-6544371da23d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:45.621963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:45.622948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:45.623445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:45.623923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:45.624753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:45.625064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:45.627449 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:45.645701 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 494/986] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:45.652516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:45.692356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:45.834942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f1fa1-4ba0-46ea-89e0-9b780617fbb1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:45.897679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:45.898931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:45.899321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:45.899648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:45.911850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:45.914547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:45.931317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:45.933051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:45.934135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 493/987] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:45.941543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:46.011373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:46.079154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fa3a6-5b33-4119-81e0-17b4d06612db None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:46.129018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41315b-9b06-45fa-9aa0-4b5f284491c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:46.129697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41315b-9b06-45fa-9aa0-4b5f284491c2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:46.129975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41315b-9b06-45fa-9aa0-4b5f284491c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:46.130246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41315b-9b06-45fa-9aa0-4b5f284491c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:46.131053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41315b-9b06-45fa-9aa0-4b5f284491c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:46.131399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41315b-9b06-45fa-9aa0-4b5f284491c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:46.132897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41315b-9b06-45fa-9aa0-4b5f284491c2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:46.148329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 495/988] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:46.155035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:46.306114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:46.336958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a4641-3eb6-4441-8ee6-90836cf072a1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:46.389995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f679a-478e-4b87-91ec-036d931671cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:46.390925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f679a-478e-4b87-91ec-036d931671cc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:46.391467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f679a-478e-4b87-91ec-036d931671cc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:46.391976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f679a-478e-4b87-91ec-036d931671cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:46.394033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f679a-478e-4b87-91ec-036d931671cc None admin] RBAC: Adding query filter params (role.id=411d0fd9d6f2463989b3b1b51a2f2c0d, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=c41cca9ec5c94473b326e0940166ee75) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:46.394610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f679a-478e-4b87-91ec-036d931671cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:46.397408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f679a-478e-4b87-91ec-036d931671cc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:46.415447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 494/989] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:46 2022] GET /identity/v3/role_assignments?user.id=c41cca9ec5c94473b326e0940166ee75&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=411d0fd9d6f2463989b3b1b51a2f2c0d => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:46.751021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a41e5-5e2f-4385-b6cb-0e82703f75fb None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:46.752172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a41e5-5e2f-4385-b6cb-0e82703f75fb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:46.752172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a41e5-5e2f-4385-b6cb-0e82703f75fb None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:46.753567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 496/990] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:46.759627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ae4af-ff9d-4845-a304-351435b3b9b6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:46.759917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ae4af-ff9d-4845-a304-351435b3b9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:46.760194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ae4af-ff9d-4845-a304-351435b3b9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:46.779055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9e0ae4af-ff9d-4845-a304-351435b3b9b6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:46.780357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9e0ae4af-ff9d-4845-a304-351435b3b9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:46.884369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 495/991] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:46.910367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a41e5-5e2f-4385-b6cb-0e82703f75fb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:47.071887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a41e5-5e2f-4385-b6cb-0e82703f75fb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:47.092776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:47.125225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:47.126051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.126335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.127073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.127836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:47.129248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:47.134269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5e064-9094-40ac-81e1-17e454e016e6 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.134556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5e064-9094-40ac-81e1-17e454e016e6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.134835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5e064-9094-40ac-81e1-17e454e016e6 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.136552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 496/992] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:47.144924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:47.147344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:47.149325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:47.152186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea49bf-5d0a-4cb3-aacb-7ab690f66cbd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.152186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea49bf-5d0a-4cb3-aacb-7ab690f66cbd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.152186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea49bf-5d0a-4cb3-aacb-7ab690f66cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.155207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 497/993] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:47.160565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:47.168732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-45ea49bf-5d0a-4cb3-aacb-7ab690f66cbd None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:47.170010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-45ea49bf-5d0a-4cb3-aacb-7ab690f66cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:47.267259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 497/994] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:47.290826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea49bf-5d0a-4cb3-aacb-7ab690f66cbd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:47.295178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:47.320004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be06cb0-d071-4329-917b-6c8f6d4bbf50 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:47.383532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:47.384349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.384854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.385362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.386663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:47.387254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:47.389860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:47.396359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 498/995] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:47.405161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:47.414053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea49bf-5d0a-4cb3-aacb-7ab690f66cbd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:47.462694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:47.463663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.463950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.464233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.465054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:47.466425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:47.474124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:47.480110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:47.481579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:47.482439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 498/996] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 1 03:44:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:47.490862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:47.571161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6770-f58f-4c5c-9508-a2a44e262ec1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:47.577445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:47.621238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:47.621939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.622209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.622485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.623213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:47.623547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:47.624953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:47.636000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 499/997] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:47.636197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9706c-6eae-4cfa-9b08-f10fc1ac4ede None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:47.637658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:47.684513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:47.685194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.685480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.685753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.686847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:47.687368 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:47.689444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:47.699933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 499/998] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:47.707885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:47.805648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d22aa-bcea-4ead-85c3-7dd9bdf0e435 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:47.808163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:47.842389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec83ec-4a05-404a-bffc-4c9463f206c0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:47.850626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:47.851431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.851735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.852037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.852836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] RBAC: Adding query filter params (interface=public, service_id=f250d60c208548f8b5938edce9d74a33, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:47.853151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:47.854590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:47.861453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 500/999] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 1 03:44:47 2022] GET /identity/v3/endpoints?service_id=f250d60c208548f8b5938edce9d74a33&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:47.866956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:47.875252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:47.881155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:47.881818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:47.882110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:47.882391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:47.883103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:47.883472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:47.884860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:47.890310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 500/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 1 03:44:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:47.896277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:48.083688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a299de6-8b05-4134-b096-5a9ae0ddc6bf None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:48.128377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a9b8b-4b4d-4374-b324-b50cc1ae7071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:48.129388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a9b8b-4b4d-4374-b324-b50cc1ae7071 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:48.129821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a9b8b-4b4d-4374-b324-b50cc1ae7071 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:48.130229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a9b8b-4b4d-4374-b324-b50cc1ae7071 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:48.131200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a9b8b-4b4d-4374-b324-b50cc1ae7071 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:48.132992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a9b8b-4b4d-4374-b324-b50cc1ae7071 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:48.140856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4526-2b22-4215-acae-2fa13024ac4a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:48.141083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 501/1001] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:44:47 2022] GET /identity/v3/services => generated 1727 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 03:44:48.203127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca592c62-aa88-4fa5-af61-b279b1fd4a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:48.204136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca592c62-aa88-4fa5-af61-b279b1fd4a5d None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:48.204653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca592c62-aa88-4fa5-af61-b279b1fd4a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:48.205111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca592c62-aa88-4fa5-af61-b279b1fd4a5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:48.205920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca592c62-aa88-4fa5-af61-b279b1fd4a5d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:48.208097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca592c62-aa88-4fa5-af61-b279b1fd4a5d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:48.315666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 501/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:44:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:44:48.611588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdca82-b1b2-4b46-a849-4cdb44c224e7 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:48.611855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdca82-b1b2-4b46-a849-4cdb44c224e7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:48.612241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdca82-b1b2-4b46-a849-4cdb44c224e7 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:48.613825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 502/1003] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:48.621697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1989938-bb8a-4e47-97ce-4442749b31e7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:48.622414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1989938-bb8a-4e47-97ce-4442749b31e7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:48.622414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1989938-bb8a-4e47-97ce-4442749b31e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:48.640514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d1989938-bb8a-4e47-97ce-4442749b31e7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:48.641722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d1989938-bb8a-4e47-97ce-4442749b31e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:48.700607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:48.724623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 502/1004] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:48 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:48.747287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdca82-b1b2-4b46-a849-4cdb44c224e7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:48.791129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:48.879052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdca82-b1b2-4b46-a849-4cdb44c224e7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:48.915848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:48.916644 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:48.916925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:48.917195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] PATH_INFO: `/v3/services/image` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:48.917854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:48.919256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:48.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:48.933640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:48.934867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:48.939619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 503/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Jul 1 03:44:48 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:48.942537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1989938-bb8a-4e47-97ce-4442749b31e7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:49.050576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1989938-bb8a-4e47-97ce-4442749b31e7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:49.088442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a80f8-7070-401c-b0e8-92ca85bf3804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:49.089120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a80f8-7070-401c-b0e8-92ca85bf3804 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:49.089404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a80f8-7070-401c-b0e8-92ca85bf3804 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:49.089679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a80f8-7070-401c-b0e8-92ca85bf3804 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:49.090406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a80f8-7070-401c-b0e8-92ca85bf3804 None admin] RBAC: Adding query filter params (name=image) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:49.090712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a80f8-7070-401c-b0e8-92ca85bf3804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:49.092118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a80f8-7070-401c-b0e8-92ca85bf3804 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:49.097787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 503/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:49.103269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:49.212521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:49.220648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a7041-cd5f-4e37-990b-9916c9fda52e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:49.259862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa63a64-e3a1-4c0b-9c79-462e6a5ec561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:49.260531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa63a64-e3a1-4c0b-9c79-462e6a5ec561 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:49.260819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa63a64-e3a1-4c0b-9c79-462e6a5ec561 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:49.261088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa63a64-e3a1-4c0b-9c79-462e6a5ec561 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:49.261809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa63a64-e3a1-4c0b-9c79-462e6a5ec561 None admin] RBAC: Adding query filter params (type=image) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:49.262116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa63a64-e3a1-4c0b-9c79-462e6a5ec561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:49.263524 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa63a64-e3a1-4c0b-9c79-462e6a5ec561 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:49.275252 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 504/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:50.726046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce00a3-558d-4816-80b5-1040e80b2769 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:50.726673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce00a3-558d-4816-80b5-1040e80b2769 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:50.726673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce00a3-558d-4816-80b5-1040e80b2769 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:50.728027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 504/1008] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:50.734605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3bca3-4e68-478a-8977-fbdebd5abb2e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:50.735154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3bca3-4e68-478a-8977-fbdebd5abb2e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:50.735395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3bca3-4e68-478a-8977-fbdebd5abb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:50.753854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-76c3bca3-4e68-478a-8977-fbdebd5abb2e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:50.755084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-76c3bca3-4e68-478a-8977-fbdebd5abb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:50.820065 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ddc6-b84e-44a6-9421-fb8f002f26b4 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:50.821260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ddc6-b84e-44a6-9421-fb8f002f26b4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:50.822279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ddc6-b84e-44a6-9421-fb8f002f26b4 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:50.827004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 505/1009] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:50.834390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28f9f2-99db-43ae-b133-35a4930a4c6b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:50.835247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 505/1010] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:50.835677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28f9f2-99db-43ae-b133-35a4930a4c6b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:50.836551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28f9f2-99db-43ae-b133-35a4930a4c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:50.859094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3bca3-4e68-478a-8977-fbdebd5abb2e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:50.866403 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6b28f9f2-99db-43ae-b133-35a4930a4c6b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:50.868416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6b28f9f2-99db-43ae-b133-35a4930a4c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:50.957258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:50.974575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:50.996520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3bca3-4e68-478a-8977-fbdebd5abb2e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:50.998669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 506/1011] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:51.027831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28f9f2-99db-43ae-b133-35a4930a4c6b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:51.037173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:51.037988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.038271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.038555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] PATH_INFO: `/v3/services/compute` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.039216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:51.040612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:51.053202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:51.054709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:51.056201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 506/1012] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:51.060965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:51.191550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8818fb1-df4a-4ed8-8d37-2129341f1aec None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:51.196689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28f9f2-99db-43ae-b133-35a4930a4c6b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:51.237655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18837b6a-fef4-4f1b-9f0b-e6343f5f2733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:51.238399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18837b6a-fef4-4f1b-9f0b-e6343f5f2733 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.238683 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18837b6a-fef4-4f1b-9f0b-e6343f5f2733 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.238922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18837b6a-fef4-4f1b-9f0b-e6343f5f2733 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.240711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18837b6a-fef4-4f1b-9f0b-e6343f5f2733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:51.241538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18837b6a-fef4-4f1b-9f0b-e6343f5f2733 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:51.248048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 507/1013] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:51.248199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:51.248199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.248199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.248199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.248487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:51.248717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:51.248918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18837b6a-fef4-4f1b-9f0b-e6343f5f2733 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:51.251845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:51.267520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 507/1014] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:51.270259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:51.303208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:51.310319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:51.388496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18837b6a-fef4-4f1b-9f0b-e6343f5f2733 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:51.417876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9d63-0268-4fa7-8280-3c6c9a96f6fe None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:51.429040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44308074-3b63-4e1b-92f1-b3ddba899907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:51.429960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44308074-3b63-4e1b-92f1-b3ddba899907 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.430241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44308074-3b63-4e1b-92f1-b3ddba899907 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.430525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44308074-3b63-4e1b-92f1-b3ddba899907 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.431249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44308074-3b63-4e1b-92f1-b3ddba899907 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:51.432788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44308074-3b63-4e1b-92f1-b3ddba899907 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:51.456312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aebe9-9f0c-4425-a98d-c8fc2581e5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:51.460335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aebe9-9f0c-4425-a98d-c8fc2581e5ee None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.460335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aebe9-9f0c-4425-a98d-c8fc2581e5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.460335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aebe9-9f0c-4425-a98d-c8fc2581e5ee None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.460335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aebe9-9f0c-4425-a98d-c8fc2581e5ee None admin] RBAC: Adding query filter params (type=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:51.460335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aebe9-9f0c-4425-a98d-c8fc2581e5ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:51.460335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aebe9-9f0c-4425-a98d-c8fc2581e5ee None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:51.465718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 508/1015] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 1 03:44:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:51.468047 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb643e-3be0-49ea-b363-a8403628b1cc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.468532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb643e-3be0-49ea-b363-a8403628b1cc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.469011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb643e-3be0-49ea-b363-a8403628b1cc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.472045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 509/1016] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:51.474186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb643e-3be0-49ea-b363-a8403628b1cc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:51.553867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 508/1017] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:44:51 2022] POST /identity/v3/projects => generated 311 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:44:51.555777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4fb42-e095-4c90-9e9c-ac6335467524 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.556083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4fb42-e095-4c90-9e9c-ac6335467524 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.556371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4fb42-e095-4c90-9e9c-ac6335467524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.564682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:51.580566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a5d4fb42-e095-4c90-9e9c-ac6335467524 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:51.581887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a5d4fb42-e095-4c90-9e9c-ac6335467524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:51.597493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb643e-3be0-49ea-b363-a8403628b1cc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:51.663003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 509/1018] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:51.668723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adf5f6-0bcb-4f57-be6e-650f87c74da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:51.669843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adf5f6-0bcb-4f57-be6e-650f87c74da4 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.670345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adf5f6-0bcb-4f57-be6e-650f87c74da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.670840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adf5f6-0bcb-4f57-be6e-650f87c74da4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.672520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adf5f6-0bcb-4f57-be6e-650f87c74da4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:51.675600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adf5f6-0bcb-4f57-be6e-650f87c74da4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:51.687387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4fb42-e095-4c90-9e9c-ac6335467524 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:51.726187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:51.751138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 510/1019] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:44:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:44:51.836596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4fb42-e095-4c90-9e9c-ac6335467524 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:51.913122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b81f04-7bb6-46a3-a58e-1d8b7dfe5444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:51.914613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81f04-7bb6-46a3-a58e-1d8b7dfe5444 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:51.915482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81f04-7bb6-46a3-a58e-1d8b7dfe5444 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:51.916220 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81f04-7bb6-46a3-a58e-1d8b7dfe5444 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:51.917815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b81f04-7bb6-46a3-a58e-1d8b7dfe5444 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:51.920422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b81f04-7bb6-46a3-a58e-1d8b7dfe5444 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:52.011649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 510/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:44:51 2022] POST /identity/v3/services => generated 242 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:44:53.750258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130b50a-edb7-42ad-ade4-46a6468e9bb9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:53.750815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130b50a-edb7-42ad-ade4-46a6468e9bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:53.750815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130b50a-edb7-42ad-ade4-46a6468e9bb9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:53.752783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 511/1021] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:53.758264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc8d65-2ae2-4daf-a820-c4f13fee1dcc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:53.758719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc8d65-2ae2-4daf-a820-c4f13fee1dcc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:53.758816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc8d65-2ae2-4daf-a820-c4f13fee1dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:53.775885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6abc8d65-2ae2-4daf-a820-c4f13fee1dcc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:53.777069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6abc8d65-2ae2-4daf-a820-c4f13fee1dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:53.856520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 511/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:53.881705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130b50a-edb7-42ad-ade4-46a6468e9bb9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:53.994636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130b50a-edb7-42ad-ade4-46a6468e9bb9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.002303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a83dc-f334-4d6c-8725-8f515b85c1a9 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.002587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a83dc-f334-4d6c-8725-8f515b85c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.002869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a83dc-f334-4d6c-8725-8f515b85c1a9 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.004454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 512/1023] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:54.009433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3d368-42b2-4a4a-9434-ee54d1f988b7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.009725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3d368-42b2-4a4a-9434-ee54d1f988b7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.010009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3d368-42b2-4a4a-9434-ee54d1f988b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.026917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-72c3d368-42b2-4a4a-9434-ee54d1f988b7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:54.028176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-72c3d368-42b2-4a4a-9434-ee54d1f988b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:54.030343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:54.037801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4f104-e7c9-4495-a253-c2ec193cd9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.038601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4f104-e7c9-4495-a253-c2ec193cd9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.038881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4f104-e7c9-4495-a253-c2ec193cd9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.039153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4f104-e7c9-4495-a253-c2ec193cd9e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.040369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4f104-e7c9-4495-a253-c2ec193cd9e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:54.042932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4f104-e7c9-4495-a253-c2ec193cd9e6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:54.047271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 512/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:54.050139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4f104-e7c9-4495-a253-c2ec193cd9e6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.055087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:54.103123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 513/1025] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:54.128563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3d368-42b2-4a4a-9434-ee54d1f988b7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.159122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4f104-e7c9-4495-a253-c2ec193cd9e6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.202128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46576e-7b8c-4dcb-a2bb-871a2b5b72f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.202128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46576e-7b8c-4dcb-a2bb-871a2b5b72f3 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.202128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46576e-7b8c-4dcb-a2bb-871a2b5b72f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.202128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46576e-7b8c-4dcb-a2bb-871a2b5b72f3 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.202612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46576e-7b8c-4dcb-a2bb-871a2b5b72f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:54.203249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46576e-7b8c-4dcb-a2bb-871a2b5b72f3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:54.245579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3d368-42b2-4a4a-9434-ee54d1f988b7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.281547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd6664-1286-48b0-8ae8-d817e8dec473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.282859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd6664-1286-48b0-8ae8-d817e8dec473 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.283185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd6664-1286-48b0-8ae8-d817e8dec473 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.283624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd6664-1286-48b0-8ae8-d817e8dec473 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.291041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:54.295006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd6664-1286-48b0-8ae8-d817e8dec473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6fbd6664-1286-48b0-8ae8-d817e8dec473 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:54.304059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:54.305306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:54.306147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 514/1026] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:54.307249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d9a2d-a341-42f0-a8c8-15831698809f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.307508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d9a2d-a341-42f0-a8c8-15831698809f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.307785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d9a2d-a341-42f0-a8c8-15831698809f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.309050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 515/1027] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:44:54.311598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d9a2d-a341-42f0-a8c8-15831698809f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.314504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 513/1028] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:44:54 2022] POST /identity/v3/users => generated 285 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:44:54.315950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54e22d-8e94-4060-a9d1-ff0220bda20d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.316252 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54e22d-8e94-4060-a9d1-ff0220bda20d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.316533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54e22d-8e94-4060-a9d1-ff0220bda20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.336618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-db54e22d-8e94-4060-a9d1-ff0220bda20d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:54.337882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-db54e22d-8e94-4060-a9d1-ff0220bda20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:54.408715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:54.422127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 514/1029] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:54.437611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d9a2d-a341-42f0-a8c8-15831698809f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.447570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54e22d-8e94-4060-a9d1-ff0220bda20d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.477522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.478198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.478483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.479072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.479606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:54.479921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:54.481335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:54.491984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 516/1030] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:54.498043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.568830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:54.577049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54e22d-8e94-4060-a9d1-ff0220bda20d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.614319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0822d9-5bed-4734-9b2b-71f91c6a0d97 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.617601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.618604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.618884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.619159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.619856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:54.621234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:54.634587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:54.635798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:54.636676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 515/1031] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:54.642836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.653008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.653817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.654103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.654379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.668263 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:54.669747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:54.689955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:54.690931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:54.691896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:54.691896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:54.691896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:54.691896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:54.691896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 517/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:54.696922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.716182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:54.769338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451e564-d12a-4492-a554-0ed697ec82b4 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.807774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a433dd0-601a-498f-9de6-51d1c4dabcf2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.810151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.810875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.811161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.811525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.812241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] RBAC: Adding query filter params (name=image) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:54.812548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:54.814490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:54.823275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 516/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:54.823451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:54.825596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.845721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.846419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.846683 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.846960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.847735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:54.848048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:54.851147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:54.867560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 518/1034] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:54.875928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:54.950338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e58ebd-2429-444d-be7b-50df3b1c2348 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.967740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:54.984477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:54.985143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:54.985947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:54.986061 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3422f-1fe2-4d32-837e-dc5971cd54a4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:54.986230 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:54.986918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] RBAC: Adding query filter params (type=image) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:54.987237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:54.988612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:54.993556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 517/1035] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:54.999432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:55.025928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:55.026646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:55.026959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:55.027187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:55.035852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:55.037290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:44:55.052696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:55.053720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:44:55.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 519/1036] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:55.059699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:55.094991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:55.114801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d02f80-7e4c-480b-86ed-2f76f86da420 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:55.153769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:55.154444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:55.154756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:55.155032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:55.155870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] RBAC: Adding query filter params (interface=public, service_id=228ffdbf8ab2471aa8f4f40864031248, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:55.156187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:55.158138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:55.168714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 518/1037] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 1 03:44:54 2022] GET /identity/v3/endpoints?service_id=228ffdbf8ab2471aa8f4f40864031248&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:55.169972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:55.222843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:55.239213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca5575-05f2-4107-bb6c-347e52f4d2a5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:55.289028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:55.289632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:55.289909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:55.290188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:55.291033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:55.291366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:55.292808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:55.305961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9e4f3-da0b-4748-a47a-7eb72cf159f2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:55.314172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 520/1038] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:44:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:55.319678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:55.366457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f039e-060b-4ce1-b88b-4446ef5a23eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:55.367598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f039e-060b-4ce1-b88b-4446ef5a23eb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:55.368085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f039e-060b-4ce1-b88b-4446ef5a23eb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:55.368572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f039e-060b-4ce1-b88b-4446ef5a23eb None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:55.369821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f039e-060b-4ce1-b88b-4446ef5a23eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:55.372257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f039e-060b-4ce1-b88b-4446ef5a23eb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:55.395258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 519/1039] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:44:55 2022] GET /identity/v3/services => generated 1957 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 03:44:55.448224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:55.462491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0c489-1d8f-432a-bcb8-64091a88083e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:55.516279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6055491-5f1b-4cab-b13c-088780ebfac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:55.518024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6055491-5f1b-4cab-b13c-088780ebfac6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:55.518521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6055491-5f1b-4cab-b13c-088780ebfac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:55.519022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6055491-5f1b-4cab-b13c-088780ebfac6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:55.635111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6055491-5f1b-4cab-b13c-088780ebfac6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:55.637170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6055491-5f1b-4cab-b13c-088780ebfac6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:55.760667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:55.781607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 521/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 03:44:55.782933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f039e-060b-4ce1-b88b-4446ef5a23eb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:55.950886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f039e-060b-4ce1-b88b-4446ef5a23eb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:55.966253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:55.999846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:56.000891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:56.001903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:56.002924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:56.022164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:56.024616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:56.056902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:56.058680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:56.061069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:56.061069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:56.061069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:56.061069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:56.061069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 520/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:56.065872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6055491-5f1b-4cab-b13c-088780ebfac6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:56.185191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6055491-5f1b-4cab-b13c-088780ebfac6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:56.224361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:56.224957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:56.225248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:56.225520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:56.226251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:56.226562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:56.227976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:56.246679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 522/1042] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:56.248248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:56.388128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef4ac-9a5f-4c75-9f7b-0b77b6286a63 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:56.433276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddfe93-2ad2-4af0-82fc-25cfcde6701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:56.434444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddfe93-2ad2-4af0-82fc-25cfcde6701c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:56.434940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddfe93-2ad2-4af0-82fc-25cfcde6701c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:56.435435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddfe93-2ad2-4af0-82fc-25cfcde6701c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:56.444191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddfe93-2ad2-4af0-82fc-25cfcde6701c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:56.446023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddfe93-2ad2-4af0-82fc-25cfcde6701c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-66ddfe93-2ad2-4af0-82fc-25cfcde6701c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:44:56.462999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:56.464054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:56.464969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:44:56.464969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:44:56.464969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:44:56.464969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:44:56.464969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:44:56.464969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:44:56.464969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:44:56.464969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:56.465396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 521/1043] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:44:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:56.473520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:56.484245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fb3c-7fbe-4b96-8cb7-2adcf510f369 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:56.484781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fb3c-7fbe-4b96-8cb7-2adcf510f369 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:56.485243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fb3c-7fbe-4b96-8cb7-2adcf510f369 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:56.487096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 522/1044] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:56.493161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ffaa-90c8-41ca-a5f3-5d56837e63a6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:56.493446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ffaa-90c8-41ca-a5f3-5d56837e63a6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:56.493814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199ffaa-90c8-41ca-a5f3-5d56837e63a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:56.524046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a199ffaa-90c8-41ca-a5f3-5d56837e63a6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:56.527286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a199ffaa-90c8-41ca-a5f3-5d56837e63a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:56.549544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:56.553114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:56.621436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876c200-affe-4422-9024-66c955d04a32 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:56.643636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 523/1045] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:56.661842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ffaa-90c8-41ca-a5f3-5d56837e63a6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:56.669137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:56.669814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:56.670097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:56.670371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:56.671188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:56.671519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:56.673067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:56.688105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 523/1046] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:44:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:56.695127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:56.807168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199ffaa-90c8-41ca-a5f3-5d56837e63a6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:56.834573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410446af-9944-4feb-b7f7-9bd9c552f0a8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:56.846227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:56.856663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395eecc-7b65-4dae-ba8e-a79d6c40b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:56.858360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395eecc-7b65-4dae-ba8e-a79d6c40b422 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:56.859120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395eecc-7b65-4dae-ba8e-a79d6c40b422 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:56.859849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395eecc-7b65-4dae-ba8e-a79d6c40b422 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:56.872659 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:56.873221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:56.873501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:56.873777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:56.874610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] RBAC: Adding query filter params (role.id=411d0fd9d6f2463989b3b1b51a2f2c0d, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=dbd31b58571b476bad12735ff7445d55) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:56.874921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:56.876988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:56.881516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395eecc-7b65-4dae-ba8e-a79d6c40b422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:56.883086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 524/1047] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:56 2022] GET /identity/v3/role_assignments?user.id=dbd31b58571b476bad12735ff7445d55&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=411d0fd9d6f2463989b3b1b51a2f2c0d => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-3395eecc-7b65-4dae-ba8e-a79d6c40b422 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:56.890860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:56.892905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:56.894270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 524/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:44:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:44:56.902930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:57.087957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea31072-4f3e-40f7-8536-17ee46a5cabb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:57.128741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:57.129490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:57.129767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:57.130042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:57.130803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:57.131108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:57.132541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:57.142502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 525/1049] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:57.148646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395eecc-7b65-4dae-ba8e-a79d6c40b422 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:57.175801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:57.321962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395eecc-7b65-4dae-ba8e-a79d6c40b422 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:57.379690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f00b2-66a9-4e10-957e-ac4f948fd315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:57.380654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f00b2-66a9-4e10-957e-ac4f948fd315 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:57.380952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f00b2-66a9-4e10-957e-ac4f948fd315 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:57.381417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f00b2-66a9-4e10-957e-ac4f948fd315 None admin] PATH_INFO: `/v3/users/7d5483d448564f529c914d1c101c8a0d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:57.405060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f00b2-66a9-4e10-957e-ac4f948fd315 None admin] RBAC: Authorizing `identity:get_user(user_id=7d5483d448564f529c914d1c101c8a0d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:57.407808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f00b2-66a9-4e10-957e-ac4f948fd315 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:57.415368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 525/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:44:57 2022] GET /identity/v3/users/7d5483d448564f529c914d1c101c8a0d => generated 285 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:57.419790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:57.481472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:57.539030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6f38-25b8-4fef-bdb7-164b4ba57447 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:57.539337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6f38-25b8-4fef-bdb7-164b4ba57447 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:57.539651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6f38-25b8-4fef-bdb7-164b4ba57447 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:57.541204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 526/1051] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:57.546542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331765b-a39b-44cc-bff0-eef0fd795e7f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:57.546916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331765b-a39b-44cc-bff0-eef0fd795e7f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:57.547079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331765b-a39b-44cc-bff0-eef0fd795e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:57.566032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7331765b-a39b-44cc-bff0-eef0fd795e7f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:57.567312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7331765b-a39b-44cc-bff0-eef0fd795e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:57.585871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82e4fb-58f7-46d0-9ebf-bc7a8187d017 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:57.625343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387e982-bff3-4b16-8fc5-8d5250072c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:57.626042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387e982-bff3-4b16-8fc5-8d5250072c12 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:57.626319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387e982-bff3-4b16-8fc5-8d5250072c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:57.626591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387e982-bff3-4b16-8fc5-8d5250072c12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:57.678302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:57.686086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 527/1052] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:44:57.713807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331765b-a39b-44cc-bff0-eef0fd795e7f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:57.725808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387e982-bff3-4b16-8fc5-8d5250072c12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:57.727378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387e982-bff3-4b16-8fc5-8d5250072c12 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:57.769316 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:57.854225 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 526/1053] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:44:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:44:57.862669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387e982-bff3-4b16-8fc5-8d5250072c12 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:57.885607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331765b-a39b-44cc-bff0-eef0fd795e7f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:57.923136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:57.923962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:57.924247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:57.924518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] PATH_INFO: `/v3/services/image` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:57.925182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:57.926553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:57.938719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:57.939783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 03:44:57.940641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:57.944921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 528/1054] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Jul 1 03:44:57 2022] GET /identity/v3/services/image => generated 86 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:57.945612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:58.015457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:58.031197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387e982-bff3-4b16-8fc5-8d5250072c12 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:58.070727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6448a9-e628-40bf-a21f-73293721a29b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:58.080729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320c685-eeec-4335-a1a7-2fe095c490ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:58.082296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c685-eeec-4335-a1a7-2fe095c490ad None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:58.083424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c685-eeec-4335-a1a7-2fe095c490ad None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:58.083926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c685-eeec-4335-a1a7-2fe095c490ad None admin] PATH_INFO: `/v3/projects/69598385b006457795f0d3dba4ad7d3f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:58.086769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320c685-eeec-4335-a1a7-2fe095c490ad None admin] RBAC: Authorizing `identity:get_project(project_id=69598385b006457795f0d3dba4ad7d3f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:58.089554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320c685-eeec-4335-a1a7-2fe095c490ad None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:58.098282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 527/1055] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:44:57 2022] GET /identity/v3/projects/69598385b006457795f0d3dba4ad7d3f => generated 311 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:58.100309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320c685-eeec-4335-a1a7-2fe095c490ad None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:58.107284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:58.113219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:58.113903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:58.114189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:58.114529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:58.115286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] RBAC: Adding query filter params (name=image) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:58.115647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:58.120541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:58.127425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 529/1056] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:58.136936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:58.224476 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320c685-eeec-4335-a1a7-2fe095c490ad None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:58.260587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:58.261248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:58.261531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:58.261866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:58.262751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=69598385b006457795f0d3dba4ad7d3f, user.id=7d5483d448564f529c914d1c101c8a0d) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:58.263068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:58.264531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:58.271256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 528/1057] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:44:58 2022] GET /identity/v3/role_assignments?user.id=7d5483d448564f529c914d1c101c8a0d&scope.project.id=69598385b006457795f0d3dba4ad7d3f&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:58.324829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a0456-9429-46f3-901c-efa7987edfad None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:58.376789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:58.391338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df8833-e845-4fd5-91a5-03aa58adb1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:58.400129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df8833-e845-4fd5-91a5-03aa58adb1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:58.400129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df8833-e845-4fd5-91a5-03aa58adb1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:58.400129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df8833-e845-4fd5-91a5-03aa58adb1f0 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:58.400129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df8833-e845-4fd5-91a5-03aa58adb1f0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:58.400129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df8833-e845-4fd5-91a5-03aa58adb1f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:58.400129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df8833-e845-4fd5-91a5-03aa58adb1f0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:58.410103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 530/1058] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:44:58.418257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:58.501636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:58.547046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c548-35c7-4387-8097-4c40eb67c8ad None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:58.586539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac54d9f-969b-4bcd-9efe-eb47d86c9817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:58.587431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac54d9f-969b-4bcd-9efe-eb47d86c9817 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:58.587742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac54d9f-969b-4bcd-9efe-eb47d86c9817 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:58.588022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac54d9f-969b-4bcd-9efe-eb47d86c9817 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:58.588776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac54d9f-969b-4bcd-9efe-eb47d86c9817 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:58.590163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac54d9f-969b-4bcd-9efe-eb47d86c9817 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:58.678059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 529/1059] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:44:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 03:44:59.015057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cfb45-2322-49e7-a26e-706128cea3e9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:59.015870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cfb45-2322-49e7-a26e-706128cea3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:59.015870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cfb45-2322-49e7-a26e-706128cea3e9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:59.017904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 531/1060] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:44:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:44:59.023428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86880ca6-f7eb-48dd-85cc-b68f513ccd09 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:59.023817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86880ca6-f7eb-48dd-85cc-b68f513ccd09 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:59.023975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86880ca6-f7eb-48dd-85cc-b68f513ccd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:59.043216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-86880ca6-f7eb-48dd-85cc-b68f513ccd09 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:44:59.046245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-86880ca6-f7eb-48dd-85cc-b68f513ccd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:44:59.158758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 530/1061] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:44:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:44:59.182610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cfb45-2322-49e7-a26e-706128cea3e9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:59.292477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cfb45-2322-49e7-a26e-706128cea3e9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:59.332163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2f5d5-a816-4a80-86ea-03f147bb5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:59.332985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f5d5-a816-4a80-86ea-03f147bb5145 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:59.333267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f5d5-a816-4a80-86ea-03f147bb5145 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:59.333554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f5d5-a816-4a80-86ea-03f147bb5145 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:59.345347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2f5d5-a816-4a80-86ea-03f147bb5145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-22c2f5d5-a816-4a80-86ea-03f147bb5145 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:44:59.353707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:59.355628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:44:59.358125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:59.361681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86880ca6-f7eb-48dd-85cc-b68f513ccd09 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:59.363835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 532/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 1 03:44:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:59.431281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:59.502794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86880ca6-f7eb-48dd-85cc-b68f513ccd09 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:59.560046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:59.561097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:59.561595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:59.561897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:59.563179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:44:59.563788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:59.566206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:59.586812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 531/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 1 03:44:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:44:59.588882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2f5d5-a816-4a80-86ea-03f147bb5145 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:44:59.686259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:44:59.772595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2f5d5-a816-4a80-86ea-03f147bb5145 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:44:59.831517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:44:59.832833 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:44:59.833337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:44:59.833793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:44:59.854360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:44:59.857016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:44:59.889835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:44:59.890882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:44:59.892567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:44:59.892567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:44:59.892567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:44:59.892567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:44:59.898994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 533/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:44:59.900295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:00.057000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9996-0154-43ac-98c6-5c4a0e06b7b9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:00.081759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:00.108217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:00.108894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:00.109178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:00.109457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:00.110244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:00.110592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:00.116567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:00.136771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 532/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:00.138324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:00.254119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286193cf-d4ed-4d8f-92e4-2741eee795f8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:00.256769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:00.296618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821183c-1d49-4730-af64-f8b71c1c4616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:00.298002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821183c-1d49-4730-af64-f8b71c1c4616 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:00.298320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821183c-1d49-4730-af64-f8b71c1c4616 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:00.298601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821183c-1d49-4730-af64-f8b71c1c4616 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:00.311537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821183c-1d49-4730-af64-f8b71c1c4616 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:00.313011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821183c-1d49-4730-af64-f8b71c1c4616 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0821183c-1d49-4730-af64-f8b71c1c4616 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:45:00.331200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:00.332293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:00.338352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 534/1066] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:45:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:45:00.345096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:00.401054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5acf4e-8baf-41cc-a6e4-038b4e1d6fdc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:00.401471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5acf4e-8baf-41cc-a6e4-038b4e1d6fdc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:00.401796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5acf4e-8baf-41cc-a6e4-038b4e1d6fdc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:00.403650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 535/1067] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:00.410201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af211de-73bb-4cd2-8169-dc5494a176cb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:00.410531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af211de-73bb-4cd2-8169-dc5494a176cb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:00.410766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af211de-73bb-4cd2-8169-dc5494a176cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:00.433271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9af211de-73bb-4cd2-8169-dc5494a176cb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:00.434483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9af211de-73bb-4cd2-8169-dc5494a176cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:00.503625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff55146-fea6-443a-a8b9-ed50c313ee57 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:00.548284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 536/1068] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:00.561732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:00.562417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:00.562698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:00.562970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:00.563892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:00.564206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:00.565649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:00.587785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 533/1069] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:45:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:00.588132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af211de-73bb-4cd2-8169-dc5494a176cb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:00.593141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:00.634090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:00.746904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52839794-310d-4b29-b55d-d3aa3b28d880 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:00.749128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:00.760532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af211de-73bb-4cd2-8169-dc5494a176cb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:00.803768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a551e7-31ad-4854-9563-17a241273a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:00.804835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a551e7-31ad-4854-9563-17a241273a4c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:00.805116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a551e7-31ad-4854-9563-17a241273a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:00.805391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a551e7-31ad-4854-9563-17a241273a4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:00.806287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a37383-e60d-41a8-a77c-86bc899672fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:00.807882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37383-e60d-41a8-a77c-86bc899672fd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:00.808708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37383-e60d-41a8-a77c-86bc899672fd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:00.809199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37383-e60d-41a8-a77c-86bc899672fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:00.817572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a551e7-31ad-4854-9563-17a241273a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c2a551e7-31ad-4854-9563-17a241273a4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:00.825348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:00.826444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:00.827330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 537/1070] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:45:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:45:00.830670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a37383-e60d-41a8-a77c-86bc899672fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:00.832713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a37383-e60d-41a8-a77c-86bc899672fd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:00.833379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a551e7-31ad-4854-9563-17a241273a4c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-91a37383-e60d-41a8-a77c-86bc899672fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:45:00.853774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:00.855084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:00.856126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:45:00.856126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:45:00.856126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:45:00.856126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:00.856126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 534/1071] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:45:00.856785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102064-fa9e-4123-967a-05989a130347 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:00.856956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102064-fa9e-4123-967a-05989a130347 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:00.857185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102064-fa9e-4123-967a-05989a130347 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:00.859271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 535/1072] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:00.861064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89102064-fa9e-4123-967a-05989a130347 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:00.971509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:00.977951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a551e7-31ad-4854-9563-17a241273a4c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:00.991895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89102064-fa9e-4123-967a-05989a130347 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:01.027723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4112646-3c2f-49d4-b660-92769711cfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.029660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4112646-3c2f-49d4-b660-92769711cfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.029948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4112646-3c2f-49d4-b660-92769711cfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.030221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4112646-3c2f-49d4-b660-92769711cfb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.030954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4112646-3c2f-49d4-b660-92769711cfb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:01.031283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4112646-3c2f-49d4-b660-92769711cfb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.032703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4112646-3c2f-49d4-b660-92769711cfb7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.037399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.038847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.039403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.039897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.041189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:01.041737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.044410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 536/1073] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:01.044639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.047422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397e95-e447-413d-a80e-7a4e12ce89da None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.047951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397e95-e447-413d-a80e-7a4e12ce89da None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.048326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397e95-e447-413d-a80e-7a4e12ce89da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.067183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 538/1074] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:45:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:45:01.067183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.072840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ae397e95-e447-413d-a80e-7a4e12ce89da None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:01.072840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ae397e95-e447-413d-a80e-7a4e12ce89da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:01.091860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:01.143312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 537/1075] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:01.144114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae397e95-e447-413d-a80e-7a4e12ce89da None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.184770 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9376c-6411-4f9b-bc56-77dbc1c7de0d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:01.227671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:01.237614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.238390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.238665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.238936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.247065 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.248605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:45:01.264356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:01.266168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:01.269329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:45:01.269329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:45:01.269329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:45:01.269329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:45:01.269329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:45:01.269329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:45:01.269329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:45:01.269329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:01.270419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae397e95-e447-413d-a80e-7a4e12ce89da None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:01.270582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 539/1076] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:45:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:01.270582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.321866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20528ac-96e3-41fc-baa6-fb0d661067ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.323472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20528ac-96e3-41fc-baa6-fb0d661067ae None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.323761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20528ac-96e3-41fc-baa6-fb0d661067ae None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.324047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20528ac-96e3-41fc-baa6-fb0d661067ae None admin] PATH_INFO: `/v3/users/7d5483d448564f529c914d1c101c8a0d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.325896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20528ac-96e3-41fc-baa6-fb0d661067ae None admin] RBAC: Authorizing `identity:get_user(user_id=7d5483d448564f529c914d1c101c8a0d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.327765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20528ac-96e3-41fc-baa6-fb0d661067ae None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.330086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 538/1077] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:01 2022] GET /identity/v3/users/7d5483d448564f529c914d1c101c8a0d => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:01.330899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20528ac-96e3-41fc-baa6-fb0d661067ae None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.378841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:01.404115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93df0d-6c75-454a-98ee-0cc41cc85d47 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:01.445207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135311b4-ab5b-4b60-8ae6-61a33e621b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.446044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135311b4-ab5b-4b60-8ae6-61a33e621b24 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.446444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135311b4-ab5b-4b60-8ae6-61a33e621b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.446725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135311b4-ab5b-4b60-8ae6-61a33e621b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.446914 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20528ac-96e3-41fc-baa6-fb0d661067ae None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:01.486304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.486985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.487295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.487581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.488372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:01.488684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.490141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.501708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:01.505397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 539/1078] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:45:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:01.506234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.529093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135311b4-ab5b-4b60-8ae6-61a33e621b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.537370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135311b4-ab5b-4b60-8ae6-61a33e621b24 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.627078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:01.637073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2315f-d6b8-430b-871e-3e2d5e5b4f06 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:01.655597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 540/1079] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:01.657581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135311b4-ab5b-4b60-8ae6-61a33e621b24 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.675751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b7cd6-19dc-49e3-87bb-b274cb23a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.676609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b7cd6-19dc-49e3-87bb-b274cb23a94f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.676888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b7cd6-19dc-49e3-87bb-b274cb23a94f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.677165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b7cd6-19dc-49e3-87bb-b274cb23a94f None admin] PATH_INFO: `/v3/projects/69598385b006457795f0d3dba4ad7d3f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.678382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b7cd6-19dc-49e3-87bb-b274cb23a94f None admin] RBAC: Authorizing `identity:get_project(project_id=69598385b006457795f0d3dba4ad7d3f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.680144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b7cd6-19dc-49e3-87bb-b274cb23a94f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.682059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 540/1080] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:01 2022] GET /identity/v3/projects/69598385b006457795f0d3dba4ad7d3f => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:01.682753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b7cd6-19dc-49e3-87bb-b274cb23a94f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.766393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:01.802219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135311b4-ab5b-4b60-8ae6-61a33e621b24 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:01.806918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b7cd6-19dc-49e3-87bb-b274cb23a94f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:01.839310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e818601-ccc9-4834-9cce-6542600b47d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.840210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e818601-ccc9-4834-9cce-6542600b47d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.840491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e818601-ccc9-4834-9cce-6542600b47d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.840765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e818601-ccc9-4834-9cce-6542600b47d1 None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/dbd31b58571b476bad12735ff7445d55/roles/411d0fd9d6f2463989b3b1b51a2f2c0d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.842969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e818601-ccc9-4834-9cce-6542600b47d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=dbd31b58571b476bad12735ff7445d55, role_id=411d0fd9d6f2463989b3b1b51a2f2c0d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.843539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:01.844328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:01.844971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e818601-ccc9-4834-9cce-6542600b47d1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.845196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:01.845522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:01.857382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:01.858833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:45:01.876660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:01.877872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:01.878766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:45:01.878766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:45:01.878766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:45:01.878766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:01.878766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 541/1081] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:01.878766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.912709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 541/1082] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:45:01 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/dbd31b58571b476bad12735ff7445d55/roles/411d0fd9d6f2463989b3b1b51a2f2c0d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:45:01.913467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e818601-ccc9-4834-9cce-6542600b47d1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:01.960647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:02.039975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c06a-040c-47a7-8bf4-a31378878534 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:02.050735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e818601-ccc9-4834-9cce-6542600b47d1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:02.064839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:02.085688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9498b60-4111-4657-abb0-da875315d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:02.086678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9498b60-4111-4657-abb0-da875315d215 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:02.086962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9498b60-4111-4657-abb0-da875315d215 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:02.087289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9498b60-4111-4657-abb0-da875315d215 None admin] PATH_INFO: `/v3/projects/69598385b006457795f0d3dba4ad7d3f/users/7d5483d448564f529c914d1c101c8a0d/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:02.090831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9498b60-4111-4657-abb0-da875315d215 None admin] RBAC: Authorizing `identity:create_grant(project_id=69598385b006457795f0d3dba4ad7d3f, user_id=7d5483d448564f529c914d1c101c8a0d, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:02.092303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9498b60-4111-4657-abb0-da875315d215 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:02.107277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b593149-0654-47b3-af77-96962675d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:02.107959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b593149-0654-47b3-af77-96962675d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:02.108243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b593149-0654-47b3-af77-96962675d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:02.108516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b593149-0654-47b3-af77-96962675d0d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:02.109244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b593149-0654-47b3-af77-96962675d0d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:02.109550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b593149-0654-47b3-af77-96962675d0d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:02.111581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b593149-0654-47b3-af77-96962675d0d5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:02.124720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 542/1083] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:02.166634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 542/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:45:01 2022] PUT /identity/v3/projects/69598385b006457795f0d3dba4ad7d3f/users/7d5483d448564f529c914d1c101c8a0d/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:45:04.254821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ccc4f-a6fc-429b-a9e7-63677bad9daa None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:04.255414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ccc4f-a6fc-429b-a9e7-63677bad9daa None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:04.255414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ccc4f-a6fc-429b-a9e7-63677bad9daa None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:04.256936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 543/1085] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:04.262454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7faca-2bcb-48ed-8825-e43345ca8f9d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:04.262953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7faca-2bcb-48ed-8825-e43345ca8f9d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:04.263158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7faca-2bcb-48ed-8825-e43345ca8f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:04.280942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e9a7faca-2bcb-48ed-8825-e43345ca8f9d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:04.282193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e9a7faca-2bcb-48ed-8825-e43345ca8f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:04.360705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 543/1086] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:45:04.385305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ccc4f-a6fc-429b-a9e7-63677bad9daa None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:04.487981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:04.495577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ccc4f-a6fc-429b-a9e7-63677bad9daa None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:04.534897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702670a-8146-4e5f-916f-c2bd067104a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:04.535807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702670a-8146-4e5f-916f-c2bd067104a2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:04.536220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702670a-8146-4e5f-916f-c2bd067104a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:04.536624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702670a-8146-4e5f-916f-c2bd067104a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:04.548747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702670a-8146-4e5f-916f-c2bd067104a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f702670a-8146-4e5f-916f-c2bd067104a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:04.558853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:04.560104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:04.561852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 544/1087] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:45:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:04.568029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7faca-2bcb-48ed-8825-e43345ca8f9d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:04.673611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7faca-2bcb-48ed-8825-e43345ca8f9d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:04.686983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:04.714498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a4085-aeb5-44d7-9b43-146911bc7e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:04.715342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a4085-aeb5-44d7-9b43-146911bc7e01 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:04.715723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a4085-aeb5-44d7-9b43-146911bc7e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:04.716174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a4085-aeb5-44d7-9b43-146911bc7e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:04.717038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a4085-aeb5-44d7-9b43-146911bc7e01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:04.717446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a4085-aeb5-44d7-9b43-146911bc7e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:04.718925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a4085-aeb5-44d7-9b43-146911bc7e01 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:04.729799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 544/1088] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:45:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:45:04.735720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702670a-8146-4e5f-916f-c2bd067104a2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:04.800813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fd4ae-7846-4d60-8bb5-6ad754d1a70f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:04.800999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fd4ae-7846-4d60-8bb5-6ad754d1a70f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:04.801285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fd4ae-7846-4d60-8bb5-6ad754d1a70f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:04.802669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 545/1089] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:45:04.808104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc998a70-f325-43df-8c28-14a680f1cac6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:04.808390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc998a70-f325-43df-8c28-14a680f1cac6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:04.808669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc998a70-f325-43df-8c28-14a680f1cac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:04.832171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dc998a70-f325-43df-8c28-14a680f1cac6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:04.833417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dc998a70-f325-43df-8c28-14a680f1cac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:04.846953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702670a-8146-4e5f-916f-c2bd067104a2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:04.885216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26967d80-bb2e-422c-8711-58c4a74d7d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:04.886453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26967d80-bb2e-422c-8711-58c4a74d7d6e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:04.886736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26967d80-bb2e-422c-8711-58c4a74d7d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:04.887003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26967d80-bb2e-422c-8711-58c4a74d7d6e None admin] PATH_INFO: `/v3/users/7d5483d448564f529c914d1c101c8a0d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:04.888388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26967d80-bb2e-422c-8711-58c4a74d7d6e None admin] RBAC: Authorizing `identity:get_user(user_id=7d5483d448564f529c914d1c101c8a0d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:04.890450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26967d80-bb2e-422c-8711-58c4a74d7d6e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:04.893213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 545/1090] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:04 2022] GET /identity/v3/users/7d5483d448564f529c914d1c101c8a0d => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:04.899623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26967d80-bb2e-422c-8711-58c4a74d7d6e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:04.912122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 546/1091] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:45:04.926690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:04.937219 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc998a70-f325-43df-8c28-14a680f1cac6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:05.009642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26967d80-bb2e-422c-8711-58c4a74d7d6e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:05.047891 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc998a70-f325-43df-8c28-14a680f1cac6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:05.049646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:05.050556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02954-e8b9-41b8-bfc2-e5c0aa0b3603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:05.051305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02954-e8b9-41b8-bfc2-e5c0aa0b3603 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:05.051535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02954-e8b9-41b8-bfc2-e5c0aa0b3603 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:05.051817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02954-e8b9-41b8-bfc2-e5c0aa0b3603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:05.087477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1240-68ac-4d5a-ac33-9e7c637adcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:05.088294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1240-68ac-4d5a-ac33-9e7c637adcaa None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:05.088579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1240-68ac-4d5a-ac33-9e7c637adcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:05.088856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1240-68ac-4d5a-ac33-9e7c637adcaa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:05.101315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e1240-68ac-4d5a-ac33-9e7c637adcaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-110e1240-68ac-4d5a-ac33-9e7c637adcaa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:05.109767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:05.110794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:05.111815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 547/1092] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 1 03:45:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:45:05.117090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1240-68ac-4d5a-ac33-9e7c637adcaa None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:05.130198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02954-e8b9-41b8-bfc2-e5c0aa0b3603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:05.131752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02954-e8b9-41b8-bfc2-e5c0aa0b3603 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:05.161043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:05.257080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1240-68ac-4d5a-ac33-9e7c637adcaa None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:05.257899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 546/1093] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 03:45:05.264045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02954-e8b9-41b8-bfc2-e5c0aa0b3603 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:05.298244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:05.300834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:05.300834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:05.300834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:05.300834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:05.300834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:05.307875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:05.311543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 548/1094] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 1 03:45:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:45:05.318535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:05.351239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:05.386993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02954-e8b9-41b8-bfc2-e5c0aa0b3603 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:05.411292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:05.423868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ee64-37ce-4cc2-ba1c-65057137f305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:05.424732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ee64-37ce-4cc2-ba1c-65057137f305 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:05.425014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ee64-37ce-4cc2-ba1c-65057137f305 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:05.425291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ee64-37ce-4cc2-ba1c-65057137f305 None admin] PATH_INFO: `/v3/projects/69598385b006457795f0d3dba4ad7d3f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:05.426534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50ee64-37ce-4cc2-ba1c-65057137f305 None admin] RBAC: Authorizing `identity:get_project(project_id=69598385b006457795f0d3dba4ad7d3f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:05.428006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50ee64-37ce-4cc2-ba1c-65057137f305 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:05.430016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 547/1095] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:05 2022] GET /identity/v3/projects/69598385b006457795f0d3dba4ad7d3f => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:45:05.437190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77e5b-628d-45f3-b599-7bba2c4a9c78 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:05.438787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ee64-37ce-4cc2-ba1c-65057137f305 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:05.477583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:05.478490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:05.478773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:05.479043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:05.491843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:05.493289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:45:05.512552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:05.513661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:05.514610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:45:05.514610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:45:05.514610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:45:05.514610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:05.514610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 549/1096] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:05.519673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:05.561737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ee64-37ce-4cc2-ba1c-65057137f305 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:05.600978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:05.601634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:05.601916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:05.602189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:05.603019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=69598385b006457795f0d3dba4ad7d3f, user.id=7d5483d448564f529c914d1c101c8a0d) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:05.603363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:05.604808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:05.611617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 548/1097] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:45:05 2022] GET /identity/v3/role_assignments?user.id=7d5483d448564f529c914d1c101c8a0d&scope.project.id=69598385b006457795f0d3dba4ad7d3f&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:05.613316 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:05.641647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ae3c3-bf06-4c2b-8b4d-28434286ec21 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:05.696111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:05.697262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:05.697759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:05.698246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:05.699243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:05.699631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:05.701116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:05.715032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 550/1098] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:05.721454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:05.784098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:05.830329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a95d-2e21-47d6-80b9-e3b79aaca34a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:05.864919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:05.865705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:05.866077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:05.866268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:05.874141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:05.875665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:45:05.890109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:05.891639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:05.892974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 549/1099] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 1 03:45:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:45:05.898085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:06.005194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb5382-7188-4119-ad67-9a0c8681cb78 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:06.039680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:06.040345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:06.040629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:06.040905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:06.041765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:06.042080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:06.043520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:06.046128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:06.061798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 551/1100] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 1 03:45:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:06.067542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:06.169045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ce444-2066-423c-9f15-202d0582b142 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:06.185499 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:06.206048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40350c-1cec-48e4-b300-76952c5b155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:06.206698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40350c-1cec-48e4-b300-76952c5b155e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:06.206982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40350c-1cec-48e4-b300-76952c5b155e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:06.207290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40350c-1cec-48e4-b300-76952c5b155e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:06.278894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40350c-1cec-48e4-b300-76952c5b155e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:06.280419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40350c-1cec-48e4-b300-76952c5b155e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:06.380721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 550/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:06.386483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:06.492863 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea083a9c-f7ab-4291-93d8-0e6e46dc59c4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:06.530758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:06.531566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:06.532251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:06.532251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:06.542929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:06.544394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:45:06.562101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:06.563872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:06.565374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:45:06.565374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:45:06.565374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:45:06.565374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:06.565374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 552/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:45:06.569535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40350c-1cec-48e4-b300-76952c5b155e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:06.587487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:06.669895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40350c-1cec-48e4-b300-76952c5b155e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:06.710489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:06.711315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:06.711494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:06.711770 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:06.712906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:06.712906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:06.714202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:06.726314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 551/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:06.731954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:06.783134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:06.837844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc5d60-4961-42d9-ab52-60e0b39b07e4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:06.872818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:06.873636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:06.873917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:06.874190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:06.882599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:06.884106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:45:06.899522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:06.900682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:06.901621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 553/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:45:06.907276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:06.997042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:07.025879 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1469041-287b-448b-8f13-32c516d29bf5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:07.064899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a5fc2-2dbe-4ced-8179-88fda44d3fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:07.065472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5fc2-2dbe-4ced-8179-88fda44d3fae None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:07.065752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5fc2-2dbe-4ced-8179-88fda44d3fae None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:07.066025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5fc2-2dbe-4ced-8179-88fda44d3fae None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:07.066806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a5fc2-2dbe-4ced-8179-88fda44d3fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:07.067116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a5fc2-2dbe-4ced-8179-88fda44d3fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:07.068562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a5fc2-2dbe-4ced-8179-88fda44d3fae None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:07.081195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 552/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:07.088058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:07.186245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1c966-bb81-4f2c-a9d5-5e7bcaa5d89b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:07.193844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:07.220011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0205aae-b86e-419d-998f-5b4cf3ab3dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:07.220656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205aae-b86e-419d-998f-5b4cf3ab3dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:07.220946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205aae-b86e-419d-998f-5b4cf3ab3dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:07.221422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205aae-b86e-419d-998f-5b4cf3ab3dc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:07.222434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0205aae-b86e-419d-998f-5b4cf3ab3dc0 None admin] RBAC: Adding query filter params (role.id=411d0fd9d6f2463989b3b1b51a2f2c0d, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=dbd31b58571b476bad12735ff7445d55) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:07.222786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0205aae-b86e-419d-998f-5b4cf3ab3dc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:07.224504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0205aae-b86e-419d-998f-5b4cf3ab3dc0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:07.230422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 554/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:45:07 2022] GET /identity/v3/role_assignments?user.id=dbd31b58571b476bad12735ff7445d55&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=411d0fd9d6f2463989b3b1b51a2f2c0d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:07.734918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f2fab-357a-412b-afbc-c26a8d7a00ff None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:07.735468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f2fab-357a-412b-afbc-c26a8d7a00ff None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:07.735580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f2fab-357a-412b-afbc-c26a8d7a00ff None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:07.736915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 553/1107] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:07.742368 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152a9d-1f41-4e91-b46b-9a24c8034b7e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:07.742821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152a9d-1f41-4e91-b46b-9a24c8034b7e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:07.742939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152a9d-1f41-4e91-b46b-9a24c8034b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:07.760288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c8152a9d-1f41-4e91-b46b-9a24c8034b7e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:07.761496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c8152a9d-1f41-4e91-b46b-9a24c8034b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:07.831374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 555/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:07.856012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f2fab-357a-412b-afbc-c26a8d7a00ff None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:07.961610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f2fab-357a-412b-afbc-c26a8d7a00ff None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:07.998270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8ea02-dfe1-4784-a5b1-cbf8e5b394de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:07.999086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ea02-dfe1-4784-a5b1-cbf8e5b394de None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:07.999405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ea02-dfe1-4784-a5b1-cbf8e5b394de None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:07.999684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ea02-dfe1-4784-a5b1-cbf8e5b394de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:08.001661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8ea02-dfe1-4784-a5b1-cbf8e5b394de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:08.003299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8ea02-dfe1-4784-a5b1-cbf8e5b394de None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:08.005343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 554/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:08.010883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152a9d-1f41-4e91-b46b-9a24c8034b7e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:08.111414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:08.114341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152a9d-1f41-4e91-b46b-9a24c8034b7e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:08.147065 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93068e55-e9eb-4c69-b8be-40c8e73743f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:08.147795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93068e55-e9eb-4c69-b8be-40c8e73743f2 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:08.148078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93068e55-e9eb-4c69-b8be-40c8e73743f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:08.148349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93068e55-e9eb-4c69-b8be-40c8e73743f2 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:08.149051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93068e55-e9eb-4c69-b8be-40c8e73743f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:08.150473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93068e55-e9eb-4c69-b8be-40c8e73743f2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:08.272861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 556/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:45:08 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:45:10.345725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f0589-bf8d-4cb6-8347-5ac0cce42709 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:10.346254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f0589-bf8d-4cb6-8347-5ac0cce42709 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:10.346254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f0589-bf8d-4cb6-8347-5ac0cce42709 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:10.347666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 555/1111] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:10.352781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7253-bd47-44f2-bf50-8f6259eebbc8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:10.353168 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7253-bd47-44f2-bf50-8f6259eebbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:10.353349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7253-bd47-44f2-bf50-8f6259eebbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:10.369727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9b8c7253-bd47-44f2-bf50-8f6259eebbc8 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:10.370917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9b8c7253-bd47-44f2-bf50-8f6259eebbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:10.437632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 557/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:10.461860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f0589-bf8d-4cb6-8347-5ac0cce42709 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:10.465646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:10.563440 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f0589-bf8d-4cb6-8347-5ac0cce42709 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:10.597723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3653c-8199-45b8-b985-1271a2504c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:10.598489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3653c-8199-45b8-b985-1271a2504c0e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:10.598774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3653c-8199-45b8-b985-1271a2504c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:10.599051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3653c-8199-45b8-b985-1271a2504c0e None admin] PATH_INFO: `/v3/roles/a3df9ed8088248bbae25abcf9937fa11` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:10.600557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3653c-8199-45b8-b985-1271a2504c0e None admin] RBAC: Authorizing `identity:get_role(role_id=a3df9ed8088248bbae25abcf9937fa11)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:10.601922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3653c-8199-45b8-b985-1271a2504c0e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:10.603322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 556/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:10 2022] GET /identity/v3/roles/a3df9ed8088248bbae25abcf9937fa11 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:10.609221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c7253-bd47-44f2-bf50-8f6259eebbc8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:10.691899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:10.706140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c7253-bd47-44f2-bf50-8f6259eebbc8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:10.740537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c016a5-8d37-4790-9d36-001c7f2e4b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:10.741394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c016a5-8d37-4790-9d36-001c7f2e4b13 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:10.741678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c016a5-8d37-4790-9d36-001c7f2e4b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:10.741956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c016a5-8d37-4790-9d36-001c7f2e4b13 None admin] PATH_INFO: `/v3/users/494a35afa7084f0dbc8ad595cb52c8f7` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:10.755792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c016a5-8d37-4790-9d36-001c7f2e4b13 None admin] RBAC: Authorizing `identity:get_user(user_id=494a35afa7084f0dbc8ad595cb52c8f7)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:10.757222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c016a5-8d37-4790-9d36-001c7f2e4b13 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:10.758874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 558/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:10 2022] GET /identity/v3/users/494a35afa7084f0dbc8ad595cb52c8f7 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:10.764657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3653c-8199-45b8-b985-1271a2504c0e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:10.832472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:10.861427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3653c-8199-45b8-b985-1271a2504c0e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:10.893623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f156-1974-4243-b5a0-de30adfe904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:10.894261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f156-1974-4243-b5a0-de30adfe904e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:10.894545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f156-1974-4243-b5a0-de30adfe904e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:10.894818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f156-1974-4243-b5a0-de30adfe904e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:10.961956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98f156-1974-4243-b5a0-de30adfe904e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:10.963425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98f156-1974-4243-b5a0-de30adfe904e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:11.030204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:11.061766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 557/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:11.068120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c016a5-8d37-4790-9d36-001c7f2e4b13 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:11.167743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c016a5-8d37-4790-9d36-001c7f2e4b13 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:11.200941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359622cd-16f2-4e7f-9163-3ae3c0c6ca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:11.201807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359622cd-16f2-4e7f-9163-3ae3c0c6ca57 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:11.202087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359622cd-16f2-4e7f-9163-3ae3c0c6ca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:11.202363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359622cd-16f2-4e7f-9163-3ae3c0c6ca57 None admin] PATH_INFO: `/v3/projects/69598385b006457795f0d3dba4ad7d3f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:11.204058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359622cd-16f2-4e7f-9163-3ae3c0c6ca57 None admin] RBAC: Authorizing `identity:get_project(project_id=69598385b006457795f0d3dba4ad7d3f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:11.205476 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359622cd-16f2-4e7f-9163-3ae3c0c6ca57 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:11.207178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 559/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:11 2022] GET /identity/v3/projects/69598385b006457795f0d3dba4ad7d3f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:11.213276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f156-1974-4243-b5a0-de30adfe904e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:11.309430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f156-1974-4243-b5a0-de30adfe904e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:11.342537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bfad5-2a4b-4aa3-b345-8158e64a8e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:11.343177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bfad5-2a4b-4aa3-b345-8158e64a8e8c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:11.343487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bfad5-2a4b-4aa3-b345-8158e64a8e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:11.343758 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bfad5-2a4b-4aa3-b345-8158e64a8e8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:11.345139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bfad5-2a4b-4aa3-b345-8158e64a8e8c None admin] RBAC: Adding query filter params (role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=69598385b006457795f0d3dba4ad7d3f, user.id=494a35afa7084f0dbc8ad595cb52c8f7) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:11.345442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bfad5-2a4b-4aa3-b345-8158e64a8e8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:11.346852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bfad5-2a4b-4aa3-b345-8158e64a8e8c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:11.352323 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 558/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:45:11 2022] GET /identity/v3/role_assignments?user.id=494a35afa7084f0dbc8ad595cb52c8f7&scope.project.id=69598385b006457795f0d3dba4ad7d3f&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:13.351694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e516b-790a-42bf-86ef-55c750ffe5cf None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:13.352246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e516b-790a-42bf-86ef-55c750ffe5cf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:13.352246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e516b-790a-42bf-86ef-55c750ffe5cf None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:13.353618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 560/1118] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:13.358834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bbb82-52c2-4653-8e41-5831baab8d2f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:13.359263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bbb82-52c2-4653-8e41-5831baab8d2f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:13.359427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bbb82-52c2-4653-8e41-5831baab8d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:13.376902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f97bbb82-52c2-4653-8e41-5831baab8d2f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:13.378085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f97bbb82-52c2-4653-8e41-5831baab8d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:13.406190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:13.443767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 559/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:13.466224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e516b-790a-42bf-86ef-55c750ffe5cf None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:13.480839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:13.569049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e516b-790a-42bf-86ef-55c750ffe5cf None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:13.602414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e96e2f-01d2-41b3-9688-08908595b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:13.603194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e96e2f-01d2-41b3-9688-08908595b9cd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:13.603510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e96e2f-01d2-41b3-9688-08908595b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:13.603790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e96e2f-01d2-41b3-9688-08908595b9cd None admin] PATH_INFO: `/v3/roles/a3df9ed8088248bbae25abcf9937fa11` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:13.605265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e96e2f-01d2-41b3-9688-08908595b9cd None admin] RBAC: Authorizing `identity:get_role(role_id=a3df9ed8088248bbae25abcf9937fa11)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:13.606620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e96e2f-01d2-41b3-9688-08908595b9cd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:13.608026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 561/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:13 2022] GET /identity/v3/roles/a3df9ed8088248bbae25abcf9937fa11 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:13.614524 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bbb82-52c2-4653-8e41-5831baab8d2f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:13.711910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bbb82-52c2-4653-8e41-5831baab8d2f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:13.745112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f62aac-864d-4dbe-ae42-d2937d2f6ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:13.745971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62aac-864d-4dbe-ae42-d2937d2f6ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:13.746259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62aac-864d-4dbe-ae42-d2937d2f6ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:13.746537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62aac-864d-4dbe-ae42-d2937d2f6ca0 None admin] PATH_INFO: `/v3/users/494a35afa7084f0dbc8ad595cb52c8f7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:13.747739 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f62aac-864d-4dbe-ae42-d2937d2f6ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=494a35afa7084f0dbc8ad595cb52c8f7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:13.749167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f62aac-864d-4dbe-ae42-d2937d2f6ca0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:13.751441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 560/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:13 2022] GET /identity/v3/users/494a35afa7084f0dbc8ad595cb52c8f7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:13.757391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e96e2f-01d2-41b3-9688-08908595b9cd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:13.836958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:13.856121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e96e2f-01d2-41b3-9688-08908595b9cd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:13.889064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742645d7-cc1f-4678-83b4-e0740fb01053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:13.889917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742645d7-cc1f-4678-83b4-e0740fb01053 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:13.890197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742645d7-cc1f-4678-83b4-e0740fb01053 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:13.890470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742645d7-cc1f-4678-83b4-e0740fb01053 None admin] PATH_INFO: `/v3/projects/69598385b006457795f0d3dba4ad7d3f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:13.891700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742645d7-cc1f-4678-83b4-e0740fb01053 None admin] RBAC: Authorizing `identity:get_project(project_id=69598385b006457795f0d3dba4ad7d3f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:13.893120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742645d7-cc1f-4678-83b4-e0740fb01053 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:13.894809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 562/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:13 2022] GET /identity/v3/projects/69598385b006457795f0d3dba4ad7d3f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:13.900902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f62aac-864d-4dbe-ae42-d2937d2f6ca0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:13.935339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:13.998787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f62aac-864d-4dbe-ae42-d2937d2f6ca0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:14.031858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28d376-2b50-4b1c-b5f3-7c975aa4f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:14.032743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28d376-2b50-4b1c-b5f3-7c975aa4f233 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:14.033025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28d376-2b50-4b1c-b5f3-7c975aa4f233 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:14.033297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28d376-2b50-4b1c-b5f3-7c975aa4f233 None admin] PATH_INFO: `/v3/projects/69598385b006457795f0d3dba4ad7d3f/users/494a35afa7084f0dbc8ad595cb52c8f7/roles/a3df9ed8088248bbae25abcf9937fa11` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:14.035502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28d376-2b50-4b1c-b5f3-7c975aa4f233 None admin] RBAC: Authorizing `identity:create_grant(project_id=69598385b006457795f0d3dba4ad7d3f, user_id=494a35afa7084f0dbc8ad595cb52c8f7, role_id=a3df9ed8088248bbae25abcf9937fa11)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:14.036926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28d376-2b50-4b1c-b5f3-7c975aa4f233 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:14.101718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 561/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:45:13 2022] PUT /identity/v3/projects/69598385b006457795f0d3dba4ad7d3f/users/494a35afa7084f0dbc8ad595cb52c8f7/roles/a3df9ed8088248bbae25abcf9937fa11 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:45:16.116897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd58e4-b60d-48be-888b-1de4979eb367 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:16.117450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd58e4-b60d-48be-888b-1de4979eb367 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:16.117450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd58e4-b60d-48be-888b-1de4979eb367 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:16.118848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 563/1124] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:16.124110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d580e-5a35-4469-88ba-86e01176db8d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:16.124564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d580e-5a35-4469-88ba-86e01176db8d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:16.124665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d580e-5a35-4469-88ba-86e01176db8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:16.142276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-207d580e-5a35-4469-88ba-86e01176db8d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:16.143511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-207d580e-5a35-4469-88ba-86e01176db8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:16.209452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 562/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:16.233194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd58e4-b60d-48be-888b-1de4979eb367 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:16.338635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd58e4-b60d-48be-888b-1de4979eb367 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:16.371711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e1309-74ad-4b75-9f97-61d375b82ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:16.372491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e1309-74ad-4b75-9f97-61d375b82ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:16.372774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e1309-74ad-4b75-9f97-61d375b82ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:16.373051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e1309-74ad-4b75-9f97-61d375b82ab5 None admin] PATH_INFO: `/v3/roles/a3df9ed8088248bbae25abcf9937fa11` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:16.374527 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e1309-74ad-4b75-9f97-61d375b82ab5 None admin] RBAC: Authorizing `identity:get_role(role_id=a3df9ed8088248bbae25abcf9937fa11)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:16.376432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e1309-74ad-4b75-9f97-61d375b82ab5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:16.377799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 564/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:16 2022] GET /identity/v3/roles/a3df9ed8088248bbae25abcf9937fa11 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:16.383628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d580e-5a35-4469-88ba-86e01176db8d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:16.407831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:16.481003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d580e-5a35-4469-88ba-86e01176db8d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:16.514000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55715b-b3a9-4c58-9edb-1cd39bdc2a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:16.514858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55715b-b3a9-4c58-9edb-1cd39bdc2a82 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:16.515137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55715b-b3a9-4c58-9edb-1cd39bdc2a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:16.515434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55715b-b3a9-4c58-9edb-1cd39bdc2a82 None admin] PATH_INFO: `/v3/users/494a35afa7084f0dbc8ad595cb52c8f7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:16.516676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55715b-b3a9-4c58-9edb-1cd39bdc2a82 None admin] RBAC: Authorizing `identity:get_user(user_id=494a35afa7084f0dbc8ad595cb52c8f7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:16.518081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55715b-b3a9-4c58-9edb-1cd39bdc2a82 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:16.519737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 563/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:16 2022] GET /identity/v3/users/494a35afa7084f0dbc8ad595cb52c8f7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:16.525516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e1309-74ad-4b75-9f97-61d375b82ab5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:16.543364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:16.623896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e1309-74ad-4b75-9f97-61d375b82ab5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:16.657041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bd7b9-7fc9-4b03-a925-9a7e60d44c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:16.657682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bd7b9-7fc9-4b03-a925-9a7e60d44c2e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:16.657966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bd7b9-7fc9-4b03-a925-9a7e60d44c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:16.658240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bd7b9-7fc9-4b03-a925-9a7e60d44c2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:16.724488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bd7b9-7fc9-4b03-a925-9a7e60d44c2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:16.725919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bd7b9-7fc9-4b03-a925-9a7e60d44c2e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:16.742550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:16.823591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 565/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:16.829470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55715b-b3a9-4c58-9edb-1cd39bdc2a82 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:16.918303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:16.927628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55715b-b3a9-4c58-9edb-1cd39bdc2a82 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:16.960705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54a1a7-5e9a-4d67-a065-6f43961e068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:16.961563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a1a7-5e9a-4d67-a065-6f43961e068c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:16.961845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a1a7-5e9a-4d67-a065-6f43961e068c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:16.962122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a1a7-5e9a-4d67-a065-6f43961e068c None admin] PATH_INFO: `/v3/projects/69598385b006457795f0d3dba4ad7d3f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:16.963329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54a1a7-5e9a-4d67-a065-6f43961e068c None admin] RBAC: Authorizing `identity:get_project(project_id=69598385b006457795f0d3dba4ad7d3f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:16.964764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54a1a7-5e9a-4d67-a065-6f43961e068c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:16.966469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 564/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:16 2022] GET /identity/v3/projects/69598385b006457795f0d3dba4ad7d3f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:16.973065 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bd7b9-7fc9-4b03-a925-9a7e60d44c2e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:17.071122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bd7b9-7fc9-4b03-a925-9a7e60d44c2e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:17.086913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:17.104840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34244259-3040-471c-8e6e-288e2b99ca41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:17.105493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244259-3040-471c-8e6e-288e2b99ca41 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:17.105781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244259-3040-471c-8e6e-288e2b99ca41 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:17.106059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244259-3040-471c-8e6e-288e2b99ca41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:17.106884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244259-3040-471c-8e6e-288e2b99ca41 None admin] RBAC: Adding query filter params (role.id=a3df9ed8088248bbae25abcf9937fa11, scope.project.id=69598385b006457795f0d3dba4ad7d3f, user.id=494a35afa7084f0dbc8ad595cb52c8f7) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:17.107199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244259-3040-471c-8e6e-288e2b99ca41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:17.108658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244259-3040-471c-8e6e-288e2b99ca41 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:17.115154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 566/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:45:16 2022] GET /identity/v3/role_assignments?user.id=494a35afa7084f0dbc8ad595cb52c8f7&scope.project.id=69598385b006457795f0d3dba4ad7d3f&role.id=a3df9ed8088248bbae25abcf9937fa11 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:19.164590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b57cd-7ade-4d7a-b84e-f3198d1bf2ad None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:19.165137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b57cd-7ade-4d7a-b84e-f3198d1bf2ad None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:19.165137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b57cd-7ade-4d7a-b84e-f3198d1bf2ad None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:19.166494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 565/1131] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:19.171710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f18c-0b2b-43a9-ba7f-c8aae7171e8c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:19.172227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f18c-0b2b-43a9-ba7f-c8aae7171e8c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:19.172705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f18c-0b2b-43a9-ba7f-c8aae7171e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:19.190393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d362f18c-0b2b-43a9-ba7f-c8aae7171e8c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:19.191608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d362f18c-0b2b-43a9-ba7f-c8aae7171e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:19.258601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 567/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:19.282479 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b57cd-7ade-4d7a-b84e-f3198d1bf2ad None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:19.384753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b57cd-7ade-4d7a-b84e-f3198d1bf2ad None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:19.417719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d03b0-d05e-49fd-b4cd-33b34aa7ffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:19.418428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d03b0-d05e-49fd-b4cd-33b34aa7ffd5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:19.418713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d03b0-d05e-49fd-b4cd-33b34aa7ffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:19.418991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d03b0-d05e-49fd-b4cd-33b34aa7ffd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:19.420189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d03b0-d05e-49fd-b4cd-33b34aa7ffd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:19.422113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d03b0-d05e-49fd-b4cd-33b34aa7ffd5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:19.423766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 566/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:45:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:19.429992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362f18c-0b2b-43a9-ba7f-c8aae7171e8c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:19.531410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362f18c-0b2b-43a9-ba7f-c8aae7171e8c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:19.542124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:19.566152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d33959-f56f-48e2-b567-5540e2c74b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:19.566807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d33959-f56f-48e2-b567-5540e2c74b25 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:19.567095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d33959-f56f-48e2-b567-5540e2c74b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:19.567410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d33959-f56f-48e2-b567-5540e2c74b25 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:19.568116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d33959-f56f-48e2-b567-5540e2c74b25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:19.569583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d33959-f56f-48e2-b567-5540e2c74b25 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:19.654322 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 568/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:45:19 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:45:21.716796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980a2b-f495-47bf-b033-12f7ebc86624 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:21.717207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980a2b-f495-47bf-b033-12f7ebc86624 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:21.717357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980a2b-f495-47bf-b033-12f7ebc86624 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:21.719311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 567/1135] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:21.724337 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b884735-7b14-4f57-bc57-d483f7fb354c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:21.724764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b884735-7b14-4f57-bc57-d483f7fb354c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:21.724903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b884735-7b14-4f57-bc57-d483f7fb354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:21.741256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7b884735-7b14-4f57-bc57-d483f7fb354c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:21.742449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7b884735-7b14-4f57-bc57-d483f7fb354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:21.807148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 569/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:21.832323 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a980a2b-f495-47bf-b033-12f7ebc86624 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:21.862162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:21.937233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a980a2b-f495-47bf-b033-12f7ebc86624 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:21.971390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710498d-d93f-4dd0-ab9b-c40f1e51347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:21.972108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710498d-d93f-4dd0-ab9b-c40f1e51347c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:21.972389 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710498d-d93f-4dd0-ab9b-c40f1e51347c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:21.972661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710498d-d93f-4dd0-ab9b-c40f1e51347c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:21.973813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710498d-d93f-4dd0-ab9b-c40f1e51347c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:21.975339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710498d-d93f-4dd0-ab9b-c40f1e51347c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:21.976892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 568/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:45:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:21.982655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b884735-7b14-4f57-bc57-d483f7fb354c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:22.070921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:22.080059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b884735-7b14-4f57-bc57-d483f7fb354c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:22.112762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b16831-9179-45fc-ac3a-df66e9f61afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:22.113450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16831-9179-45fc-ac3a-df66e9f61afe None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:22.113733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16831-9179-45fc-ac3a-df66e9f61afe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:22.114011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b16831-9179-45fc-ac3a-df66e9f61afe None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:22.114714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b16831-9179-45fc-ac3a-df66e9f61afe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:22.116161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b16831-9179-45fc-ac3a-df66e9f61afe None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:22.225567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 570/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:45:21 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:45:24.315565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b010cd-914d-41bf-b383-8af79494fe38 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:24.316119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b010cd-914d-41bf-b383-8af79494fe38 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:24.316227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b010cd-914d-41bf-b383-8af79494fe38 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:24.317544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 569/1139] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:24.322916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3c80a-30d2-4457-a439-9e5e2aeea43e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:24.323363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3c80a-30d2-4457-a439-9e5e2aeea43e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:24.323518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3c80a-30d2-4457-a439-9e5e2aeea43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:24.340531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-bbb3c80a-30d2-4457-a439-9e5e2aeea43e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:24.341712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bbb3c80a-30d2-4457-a439-9e5e2aeea43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:24.406943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 571/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:24.431360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b010cd-914d-41bf-b383-8af79494fe38 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:24.525944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:24.532689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b010cd-914d-41bf-b383-8af79494fe38 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:24.567267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a041a-f235-42fb-a9bc-8ad642f2f0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:24.568114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a041a-f235-42fb-a9bc-8ad642f2f0ba None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:24.568403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a041a-f235-42fb-a9bc-8ad642f2f0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:24.568679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a041a-f235-42fb-a9bc-8ad642f2f0ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:24.579218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a041a-f235-42fb-a9bc-8ad642f2f0ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2a1a041a-f235-42fb-a9bc-8ad642f2f0ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:24.586752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:24.587829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:24.588924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 570/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:45:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:24.593995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3c80a-30d2-4457-a439-9e5e2aeea43e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:24.669393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:24.691162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3c80a-30d2-4457-a439-9e5e2aeea43e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:24.724651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:24.725322 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:24.725607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:24.725888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:24.726629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:24.726942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:24.728367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:24.737475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 572/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:45:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:24.742693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a041a-f235-42fb-a9bc-8ad642f2f0ba None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:24.840590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a041a-f235-42fb-a9bc-8ad642f2f0ba None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:24.872950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a2de8-c650-45d6-beb6-5b4a50005058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:24.873803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a2de8-c650-45d6-beb6-5b4a50005058 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:24.874084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a2de8-c650-45d6-beb6-5b4a50005058 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:24.877317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a2de8-c650-45d6-beb6-5b4a50005058 None admin] PATH_INFO: `/v3/users/7f10c47d8e96493ca584142c7182a126` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:24.888038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a2de8-c650-45d6-beb6-5b4a50005058 None admin] RBAC: Authorizing `identity:get_user(user_id=7f10c47d8e96493ca584142c7182a126)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:24.889464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a2de8-c650-45d6-beb6-5b4a50005058 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:24.891138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 571/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:24 2022] GET /identity/v3/users/7f10c47d8e96493ca584142c7182a126 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:24.896892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:24.995576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca45ae-b167-40f7-bb14-7490bcdc982f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:25.028704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17736832-4164-4d7a-83a4-361e864fe72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:25.029345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17736832-4164-4d7a-83a4-361e864fe72c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:25.029625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17736832-4164-4d7a-83a4-361e864fe72c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:25.029900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17736832-4164-4d7a-83a4-361e864fe72c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:25.040110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:25.096033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17736832-4164-4d7a-83a4-361e864fe72c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:25.097386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17736832-4164-4d7a-83a4-361e864fe72c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:25.192967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 573/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:25.199037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a2de8-c650-45d6-beb6-5b4a50005058 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:25.214278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:25.299106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a2de8-c650-45d6-beb6-5b4a50005058 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:25.332603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b9f79-f650-4d3b-8cb2-e2f7cf72078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:25.333457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b9f79-f650-4d3b-8cb2-e2f7cf72078f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:25.333738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b9f79-f650-4d3b-8cb2-e2f7cf72078f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:25.334012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b9f79-f650-4d3b-8cb2-e2f7cf72078f None admin] PATH_INFO: `/v3/projects/ea1ed4702aab410e93ca903d1b688963` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:25.335184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b9f79-f650-4d3b-8cb2-e2f7cf72078f None admin] RBAC: Authorizing `identity:get_project(project_id=ea1ed4702aab410e93ca903d1b688963)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:25.336653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b9f79-f650-4d3b-8cb2-e2f7cf72078f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:25.338386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 572/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:25 2022] GET /identity/v3/projects/ea1ed4702aab410e93ca903d1b688963 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:25.345104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17736832-4164-4d7a-83a4-361e864fe72c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:25.391908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:25.449097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17736832-4164-4d7a-83a4-361e864fe72c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:25.491411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dc5bb-7f68-46a0-9926-6dced4ffd3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:25.492531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc5bb-7f68-46a0-9926-6dced4ffd3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:25.493030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc5bb-7f68-46a0-9926-6dced4ffd3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:25.493525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc5bb-7f68-46a0-9926-6dced4ffd3a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:25.495130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dc5bb-7f68-46a0-9926-6dced4ffd3a0 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=ea1ed4702aab410e93ca903d1b688963, user.id=7f10c47d8e96493ca584142c7182a126) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:25.495740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dc5bb-7f68-46a0-9926-6dced4ffd3a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:25.498467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dc5bb-7f68-46a0-9926-6dced4ffd3a0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:25.508002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 574/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:45:25 2022] GET /identity/v3/role_assignments?user.id=7f10c47d8e96493ca584142c7182a126&scope.project.id=ea1ed4702aab410e93ca903d1b688963&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:27.522501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf3b1-41c9-47b3-b0a3-4f05b3ab206e None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:27.523033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf3b1-41c9-47b3-b0a3-4f05b3ab206e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:27.523033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf3b1-41c9-47b3-b0a3-4f05b3ab206e None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:27.524451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 573/1147] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:27.529656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6d380-b67e-4965-8bff-77c0e67c2ae0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:27.530052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6d380-b67e-4965-8bff-77c0e67c2ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:27.530221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6d380-b67e-4965-8bff-77c0e67c2ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:27.546948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-82a6d380-b67e-4965-8bff-77c0e67c2ae0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:27.548198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-82a6d380-b67e-4965-8bff-77c0e67c2ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:27.613351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 575/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:27.636244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abf3b1-41c9-47b3-b0a3-4f05b3ab206e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:27.720686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:27.739392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abf3b1-41c9-47b3-b0a3-4f05b3ab206e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:27.773354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d49fb-d078-4f43-9d0c-89636897a99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:27.774061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d49fb-d078-4f43-9d0c-89636897a99c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:27.774342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d49fb-d078-4f43-9d0c-89636897a99c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:27.774618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d49fb-d078-4f43-9d0c-89636897a99c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:27.784420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d49fb-d078-4f43-9d0c-89636897a99c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-559d49fb-d078-4f43-9d0c-89636897a99c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:27.791872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:27.792950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:27.793873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 574/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:27.799725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6d380-b67e-4965-8bff-77c0e67c2ae0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:27.837697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:27.901462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6d380-b67e-4965-8bff-77c0e67c2ae0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:27.934600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:27.935288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:27.935578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:27.935854 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:27.936594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:27.937449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:27.938809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:27.946746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 576/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:45:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:27.952894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d49fb-d078-4f43-9d0c-89636897a99c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:28.051205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d49fb-d078-4f43-9d0c-89636897a99c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:28.083703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8102a4a-09dc-4a24-821d-d55f34321f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:28.084557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8102a4a-09dc-4a24-821d-d55f34321f24 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:28.084838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8102a4a-09dc-4a24-821d-d55f34321f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:28.085112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8102a4a-09dc-4a24-821d-d55f34321f24 None admin] PATH_INFO: `/v3/users/7f10c47d8e96493ca584142c7182a126` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:28.086282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8102a4a-09dc-4a24-821d-d55f34321f24 None admin] RBAC: Authorizing `identity:get_user(user_id=7f10c47d8e96493ca584142c7182a126)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:28.087733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8102a4a-09dc-4a24-821d-d55f34321f24 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:28.089346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 575/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:27 2022] GET /identity/v3/users/7f10c47d8e96493ca584142c7182a126 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:28.095813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:28.189372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:28.195119 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d778f-598a-43aa-8f07-2fd72e9a5b30 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:28.229028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c179722-4f63-4490-b147-4249f53af420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:28.229885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c179722-4f63-4490-b147-4249f53af420 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:28.230168 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c179722-4f63-4490-b147-4249f53af420 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:28.230445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c179722-4f63-4490-b147-4249f53af420 None admin] PATH_INFO: `/v3/projects/ea1ed4702aab410e93ca903d1b688963` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:28.231648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c179722-4f63-4490-b147-4249f53af420 None admin] RBAC: Authorizing `identity:get_project(project_id=ea1ed4702aab410e93ca903d1b688963)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:28.233065 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c179722-4f63-4490-b147-4249f53af420 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:28.235411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 577/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:28 2022] GET /identity/v3/projects/ea1ed4702aab410e93ca903d1b688963 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:28.241347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8102a4a-09dc-4a24-821d-d55f34321f24 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:28.254041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:28.340910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8102a4a-09dc-4a24-821d-d55f34321f24 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:28.375167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16230d-4001-4a11-9fa6-48d4c89db0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:28.376074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16230d-4001-4a11-9fa6-48d4c89db0e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:28.376356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16230d-4001-4a11-9fa6-48d4c89db0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:28.376631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16230d-4001-4a11-9fa6-48d4c89db0e4 None admin] PATH_INFO: `/v3/projects/ea1ed4702aab410e93ca903d1b688963/users/7f10c47d8e96493ca584142c7182a126/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:28.378636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16230d-4001-4a11-9fa6-48d4c89db0e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea1ed4702aab410e93ca903d1b688963, user_id=7f10c47d8e96493ca584142c7182a126, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:28.380092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16230d-4001-4a11-9fa6-48d4c89db0e4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:28.471091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 576/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:45:28 2022] PUT /identity/v3/projects/ea1ed4702aab410e93ca903d1b688963/users/7f10c47d8e96493ca584142c7182a126/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:45:30.506037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9631f-c7f7-4ef8-bc8c-d177a25ea52a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:30.506567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9631f-c7f7-4ef8-bc8c-d177a25ea52a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:30.506567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9631f-c7f7-4ef8-bc8c-d177a25ea52a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:30.508000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 578/1154] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:30.513279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18c6f-a630-45e4-95b8-ad106bdb26e4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:30.513703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18c6f-a630-45e4-95b8-ad106bdb26e4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:30.513828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18c6f-a630-45e4-95b8-ad106bdb26e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:30.530350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f3c18c6f-a630-45e4-95b8-ad106bdb26e4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:30.532065 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f3c18c6f-a630-45e4-95b8-ad106bdb26e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:30.596002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:30.600525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 577/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:30.625500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9631f-c7f7-4ef8-bc8c-d177a25ea52a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:30.726580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9631f-c7f7-4ef8-bc8c-d177a25ea52a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:30.758980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6f49-745e-4277-997b-374f3d8dbcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:30.759724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6f49-745e-4277-997b-374f3d8dbcef None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:30.760012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6f49-745e-4277-997b-374f3d8dbcef None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:30.760291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6f49-745e-4277-997b-374f3d8dbcef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:30.770635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6f49-745e-4277-997b-374f3d8dbcef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-958c6f49-745e-4277-997b-374f3d8dbcef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:30.778205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:30.779327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:30.780390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 579/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:45:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:30.786174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c18c6f-a630-45e4-95b8-ad106bdb26e4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:30.882579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c18c6f-a630-45e4-95b8-ad106bdb26e4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:30.916990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:30.917647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:30.917929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:30.918203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:30.918955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:30.919285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:30.920683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:30.928653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 578/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:45:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:30.934720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6f49-745e-4277-997b-374f3d8dbcef None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:30.960063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:31.033382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6f49-745e-4277-997b-374f3d8dbcef None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:31.066561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556b2aa-28fa-4a5b-90e9-9989e00d9a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:31.067457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b2aa-28fa-4a5b-90e9-9989e00d9a86 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:31.067740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b2aa-28fa-4a5b-90e9-9989e00d9a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:31.068014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b2aa-28fa-4a5b-90e9-9989e00d9a86 None admin] PATH_INFO: `/v3/users/7f10c47d8e96493ca584142c7182a126` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:31.069202 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556b2aa-28fa-4a5b-90e9-9989e00d9a86 None admin] RBAC: Authorizing `identity:get_user(user_id=7f10c47d8e96493ca584142c7182a126)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:31.070628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556b2aa-28fa-4a5b-90e9-9989e00d9a86 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:31.072293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 580/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:30 2022] GET /identity/v3/users/7f10c47d8e96493ca584142c7182a126 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:31.078970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:31.140867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:31.178460 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf28046-26aa-4ce1-afc1-0c31f5b28c6d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:31.212225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e683b4-cb3d-4ec9-9ddb-3ac75da36841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:31.212864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e683b4-cb3d-4ec9-9ddb-3ac75da36841 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:31.213146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e683b4-cb3d-4ec9-9ddb-3ac75da36841 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:31.213426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e683b4-cb3d-4ec9-9ddb-3ac75da36841 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:31.280675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e683b4-cb3d-4ec9-9ddb-3ac75da36841 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:31.282108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e683b4-cb3d-4ec9-9ddb-3ac75da36841 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:31.340499 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:31.378226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 579/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:31.384131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556b2aa-28fa-4a5b-90e9-9989e00d9a86 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:31.473618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:31.481875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556b2aa-28fa-4a5b-90e9-9989e00d9a86 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:31.515340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50850afa-e332-4d39-8232-c9c00e5c8e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:31.516188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850afa-e332-4d39-8232-c9c00e5c8e6a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:31.516467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850afa-e332-4d39-8232-c9c00e5c8e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:31.516740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850afa-e332-4d39-8232-c9c00e5c8e6a None admin] PATH_INFO: `/v3/projects/ea1ed4702aab410e93ca903d1b688963` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:31.517904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50850afa-e332-4d39-8232-c9c00e5c8e6a None admin] RBAC: Authorizing `identity:get_project(project_id=ea1ed4702aab410e93ca903d1b688963)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:31.519340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50850afa-e332-4d39-8232-c9c00e5c8e6a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:31.521028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 581/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:31 2022] GET /identity/v3/projects/ea1ed4702aab410e93ca903d1b688963 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:31.527480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e683b4-cb3d-4ec9-9ddb-3ac75da36841 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:31.626382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e683b4-cb3d-4ec9-9ddb-3ac75da36841 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:31.661562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6ef5a-411c-4d72-ae0d-4b09260cce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:31.662217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ef5a-411c-4d72-ae0d-4b09260cce3c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:31.662498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ef5a-411c-4d72-ae0d-4b09260cce3c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:31.662772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ef5a-411c-4d72-ae0d-4b09260cce3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:31.663645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6ef5a-411c-4d72-ae0d-4b09260cce3c None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=ea1ed4702aab410e93ca903d1b688963, user.id=7f10c47d8e96493ca584142c7182a126) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:31.663960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6ef5a-411c-4d72-ae0d-4b09260cce3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:31.665384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6ef5a-411c-4d72-ae0d-4b09260cce3c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:31.672097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 580/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:45:31 2022] GET /identity/v3/role_assignments?user.id=7f10c47d8e96493ca584142c7182a126&scope.project.id=ea1ed4702aab410e93ca903d1b688963&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:33.717557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0aecd-ef35-44c5-be17-96ce32bc9e83 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:33.718570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0aecd-ef35-44c5-be17-96ce32bc9e83 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:33.718666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0aecd-ef35-44c5-be17-96ce32bc9e83 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:33.720047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 582/1162] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:33.725050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fde57-e65f-489d-a9d6-eeda5ab182b8 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:33.725438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fde57-e65f-489d-a9d6-eeda5ab182b8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:33.725608 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fde57-e65f-489d-a9d6-eeda5ab182b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:33.741931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-079fde57-e65f-489d-a9d6-eeda5ab182b8 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:33.743118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-079fde57-e65f-489d-a9d6-eeda5ab182b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:33.762760 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:33.809037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 581/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:33.832282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0aecd-ef35-44c5-be17-96ce32bc9e83 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:33.932602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0aecd-ef35-44c5-be17-96ce32bc9e83 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:33.965279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a4614-0780-423f-9ea7-8c9caee18158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:33.965849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a4614-0780-423f-9ea7-8c9caee18158 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:33.966131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a4614-0780-423f-9ea7-8c9caee18158 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:33.966409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a4614-0780-423f-9ea7-8c9caee18158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:33.993121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:34.033238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a4614-0780-423f-9ea7-8c9caee18158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:34.034676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a4614-0780-423f-9ea7-8c9caee18158 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:34.135547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 583/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:34.142075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fde57-e65f-489d-a9d6-eeda5ab182b8 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:34.237667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fde57-e65f-489d-a9d6-eeda5ab182b8 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:34.271039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63ba3a-ee21-4acf-aaf7-7acbf0a48ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:34.271860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ba3a-ee21-4acf-aaf7-7acbf0a48ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:34.272140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ba3a-ee21-4acf-aaf7-7acbf0a48ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:34.272409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ba3a-ee21-4acf-aaf7-7acbf0a48ad6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:34.278942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63ba3a-ee21-4acf-aaf7-7acbf0a48ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:34.280438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63ba3a-ee21-4acf-aaf7-7acbf0a48ad6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ba63ba3a-ee21-4acf-aaf7-7acbf0a48ad6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:34.294169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:34.295456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:34.296689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 582/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:45:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:34.301393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a4614-0780-423f-9ea7-8c9caee18158 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:34.361298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:34.402522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a4614-0780-423f-9ea7-8c9caee18158 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:34.435862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c1b1-114a-4345-bbba-acbc9b0793de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:34.436527 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c1b1-114a-4345-bbba-acbc9b0793de None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:34.437408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c1b1-114a-4345-bbba-acbc9b0793de None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:34.437692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c1b1-114a-4345-bbba-acbc9b0793de None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:34.438478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011c1b1-114a-4345-bbba-acbc9b0793de None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:34.438790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011c1b1-114a-4345-bbba-acbc9b0793de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:34.440182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011c1b1-114a-4345-bbba-acbc9b0793de None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:34.447778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 584/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 1 03:45:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:36.471057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608b80-ccc7-484d-bfa6-be30a2783283 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:36.471641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608b80-ccc7-484d-bfa6-be30a2783283 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:36.471641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608b80-ccc7-484d-bfa6-be30a2783283 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:36.472994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 583/1167] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:36.478200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f064f-c20d-448a-92d5-e462845f3d4e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:36.478629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f064f-c20d-448a-92d5-e462845f3d4e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:36.478756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f064f-c20d-448a-92d5-e462845f3d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:36.495809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a89f064f-c20d-448a-92d5-e462845f3d4e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:36.496998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a89f064f-c20d-448a-92d5-e462845f3d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:36.563408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 585/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:36.587838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c608b80-ccc7-484d-bfa6-be30a2783283 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:36.606859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:36.689529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c608b80-ccc7-484d-bfa6-be30a2783283 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:36.723708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de92529-0d47-4de2-b9d8-fceaec5a817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:36.724306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de92529-0d47-4de2-b9d8-fceaec5a817f None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:36.724588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de92529-0d47-4de2-b9d8-fceaec5a817f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:36.724862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de92529-0d47-4de2-b9d8-fceaec5a817f None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:36.725593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de92529-0d47-4de2-b9d8-fceaec5a817f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:36.726944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de92529-0d47-4de2-b9d8-fceaec5a817f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:36.787814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 584/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Fri Jul 1 03:45:36 2022] POST /identity/v3/domains => generated 267 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:45:38.850610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416afb-7764-4d5f-8207-600d067edca2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:38.851151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416afb-7764-4d5f-8207-600d067edca2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:38.851151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416afb-7764-4d5f-8207-600d067edca2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:38.852587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 586/1170] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:38.857710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3f98-e07f-428a-8268-8d4618dc30d1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:38.858102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3f98-e07f-428a-8268-8d4618dc30d1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:38.858261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3f98-e07f-428a-8268-8d4618dc30d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:38.875014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7e3a3f98-e07f-428a-8268-8d4618dc30d1 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:38.876514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7e3a3f98-e07f-428a-8268-8d4618dc30d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:38.944137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 585/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:38.968184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a416afb-7764-4d5f-8207-600d067edca2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:39.027017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:39.071877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a416afb-7764-4d5f-8207-600d067edca2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:39.103761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05358b-7e5f-4cd9-bf6c-8d0f7a7fe073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:39.104606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05358b-7e5f-4cd9-bf6c-8d0f7a7fe073 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:39.104888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05358b-7e5f-4cd9-bf6c-8d0f7a7fe073 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:39.105164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05358b-7e5f-4cd9-bf6c-8d0f7a7fe073 None admin] PATH_INFO: `/v3/domains/51a63a652f7440f0a7bf06cae7fb8ed0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:39.116238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05358b-7e5f-4cd9-bf6c-8d0f7a7fe073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=51a63a652f7440f0a7bf06cae7fb8ed0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:39.117666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05358b-7e5f-4cd9-bf6c-8d0f7a7fe073 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:39.119321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 587/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Fri Jul 1 03:45:38 2022] GET /identity/v3/domains/51a63a652f7440f0a7bf06cae7fb8ed0 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:39.125875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a3f98-e07f-428a-8268-8d4618dc30d1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:39.134027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:39.222599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a3f98-e07f-428a-8268-8d4618dc30d1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:39.256531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0402-9eee-45f1-8c11-25e5995c9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:39.257187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0402-9eee-45f1-8c11-25e5995c9f78 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:39.257475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0402-9eee-45f1-8c11-25e5995c9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:39.257751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0402-9eee-45f1-8c11-25e5995c9f78 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:39.258466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b0402-9eee-45f1-8c11-25e5995c9f78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:39.259917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b0402-9eee-45f1-8c11-25e5995c9f78 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:39.374490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 586/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:45:39 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:45:41.440710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf320c6-7db2-4cc9-9c05-af4a403bd977 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:41.441261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf320c6-7db2-4cc9-9c05-af4a403bd977 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:41.441261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf320c6-7db2-4cc9-9c05-af4a403bd977 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:41.442610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 588/1174] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:41.447994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d5f1-ba5f-4a67-b46d-678a7e9303fa None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:41.448433 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d5f1-ba5f-4a67-b46d-678a7e9303fa None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:41.448558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d5f1-ba5f-4a67-b46d-678a7e9303fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:41.465058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8795d5f1-ba5f-4a67-b46d-678a7e9303fa None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:41.466266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8795d5f1-ba5f-4a67-b46d-678a7e9303fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:41.521047 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:41.534800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 587/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:41.561353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf320c6-7db2-4cc9-9c05-af4a403bd977 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:41.659704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf320c6-7db2-4cc9-9c05-af4a403bd977 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:41.673636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:41.693358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0baaf6-af8e-4c81-8e92-4ff155363784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:41.694147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0baaf6-af8e-4c81-8e92-4ff155363784 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:41.694427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0baaf6-af8e-4c81-8e92-4ff155363784 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:41.694699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0baaf6-af8e-4c81-8e92-4ff155363784 None admin] PATH_INFO: `/v3/domains/51a63a652f7440f0a7bf06cae7fb8ed0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:41.695938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0baaf6-af8e-4c81-8e92-4ff155363784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=51a63a652f7440f0a7bf06cae7fb8ed0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:41.697343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0baaf6-af8e-4c81-8e92-4ff155363784 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:41.698939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 589/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Fri Jul 1 03:45:41 2022] GET /identity/v3/domains/51a63a652f7440f0a7bf06cae7fb8ed0 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:41.705301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d5f1-ba5f-4a67-b46d-678a7e9303fa None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:41.800215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d5f1-ba5f-4a67-b46d-678a7e9303fa None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:41.833726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad971c-222d-4e9a-a4e2-59773ba0e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:41.834387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad971c-222d-4e9a-a4e2-59773ba0e650 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:41.834663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad971c-222d-4e9a-a4e2-59773ba0e650 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:41.834933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad971c-222d-4e9a-a4e2-59773ba0e650 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:41.835672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad971c-222d-4e9a-a4e2-59773ba0e650 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:41.837086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad971c-222d-4e9a-a4e2-59773ba0e650 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:41.944806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 588/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:45:41 2022] POST /identity/v3/users => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:45:44.025111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ff66-311c-42fa-8157-4edd43c16257 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:44.025656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ff66-311c-42fa-8157-4edd43c16257 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:44.025656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ff66-311c-42fa-8157-4edd43c16257 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:44.027032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 590/1178] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:44.032749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fabb-0313-4978-962e-5249f8409532 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:44.033173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fabb-0313-4978-962e-5249f8409532 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:44.033306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fabb-0313-4978-962e-5249f8409532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:44.050438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d017fabb-0313-4978-962e-5249f8409532 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:44.051669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d017fabb-0313-4978-962e-5249f8409532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:44.093175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:44.119209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 589/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:44.142963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3ff66-311c-42fa-8157-4edd43c16257 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:44.240625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3ff66-311c-42fa-8157-4edd43c16257 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:44.273306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d54c37-ed7e-45cc-acdd-2d4040eb23e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:44.274077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d54c37-ed7e-45cc-acdd-2d4040eb23e6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:44.274357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d54c37-ed7e-45cc-acdd-2d4040eb23e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:44.274630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d54c37-ed7e-45cc-acdd-2d4040eb23e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:44.284827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d54c37-ed7e-45cc-acdd-2d4040eb23e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-06d54c37-ed7e-45cc-acdd-2d4040eb23e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:44.292188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:44.293303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:44.294187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 591/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:45:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:44.299543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fabb-0313-4978-962e-5249f8409532 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:44.395633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fabb-0313-4978-962e-5249f8409532 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:44.428759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:44.429685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:44.429966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:44.430238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:44.430982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:44.431314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:44.432681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:44.441154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 590/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:45:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:44.446897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d54c37-ed7e-45cc-acdd-2d4040eb23e6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:44.489016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:44.544089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d54c37-ed7e-45cc-acdd-2d4040eb23e6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:44.577028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af96106-9f8b-4774-8fbd-4e38dbc1439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:44.577888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af96106-9f8b-4774-8fbd-4e38dbc1439c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:44.578171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af96106-9f8b-4774-8fbd-4e38dbc1439c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:44.578442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af96106-9f8b-4774-8fbd-4e38dbc1439c None admin] PATH_INFO: `/v3/users/b410464de0ea4d89ab7cd1ad447b9552` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:44.591279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af96106-9f8b-4774-8fbd-4e38dbc1439c None admin] RBAC: Authorizing `identity:get_user(user_id=b410464de0ea4d89ab7cd1ad447b9552)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:44.593236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af96106-9f8b-4774-8fbd-4e38dbc1439c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:44.594971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 592/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:44 2022] GET /identity/v3/users/b410464de0ea4d89ab7cd1ad447b9552 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:44.600760 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:44.635766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:44.700444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9d8db-ce9c-403e-a08b-33ab8aaf42a1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:44.734841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7392c-b7d8-4466-9df0-b33cbd4a6894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:44.735521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7392c-b7d8-4466-9df0-b33cbd4a6894 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:44.735803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7392c-b7d8-4466-9df0-b33cbd4a6894 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:44.736078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7392c-b7d8-4466-9df0-b33cbd4a6894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:44.802732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7392c-b7d8-4466-9df0-b33cbd4a6894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:44.804131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7392c-b7d8-4466-9df0-b33cbd4a6894 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:44.836672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:44.901508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 591/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:44.908015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af96106-9f8b-4774-8fbd-4e38dbc1439c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:45.002344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:45.005277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af96106-9f8b-4774-8fbd-4e38dbc1439c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:45.039730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5094b9a-ff62-43fc-a0c2-f65cf5883a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:45.040579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094b9a-ff62-43fc-a0c2-f65cf5883a51 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:45.040865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094b9a-ff62-43fc-a0c2-f65cf5883a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:45.041139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094b9a-ff62-43fc-a0c2-f65cf5883a51 None admin] PATH_INFO: `/v3/projects/4457a4c964a44aa19674aa94d82da9f2` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:45.042323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5094b9a-ff62-43fc-a0c2-f65cf5883a51 None admin] RBAC: Authorizing `identity:get_project(project_id=4457a4c964a44aa19674aa94d82da9f2)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:45.043794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5094b9a-ff62-43fc-a0c2-f65cf5883a51 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:45.045982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 593/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:44 2022] GET /identity/v3/projects/4457a4c964a44aa19674aa94d82da9f2 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:45.052107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7392c-b7d8-4466-9df0-b33cbd4a6894 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:45.152324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7392c-b7d8-4466-9df0-b33cbd4a6894 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:45.185282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfd6fa-9dc1-476d-8c58-19fbd3ffdb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:45.185916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd6fa-9dc1-476d-8c58-19fbd3ffdb7d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:45.186751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd6fa-9dc1-476d-8c58-19fbd3ffdb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:45.187022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd6fa-9dc1-476d-8c58-19fbd3ffdb7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:45.187873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfd6fa-9dc1-476d-8c58-19fbd3ffdb7d None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=4457a4c964a44aa19674aa94d82da9f2, user.id=b410464de0ea4d89ab7cd1ad447b9552) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:45.188180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfd6fa-9dc1-476d-8c58-19fbd3ffdb7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:45.189600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfd6fa-9dc1-476d-8c58-19fbd3ffdb7d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:45.192472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:45.195340 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 592/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:45:45 2022] GET /identity/v3/role_assignments?user.id=b410464de0ea4d89ab7cd1ad447b9552&scope.project.id=4457a4c964a44aa19674aa94d82da9f2&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:47.181710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bec67b-00ff-42fe-b74b-681f5668d422 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:47.182283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bec67b-00ff-42fe-b74b-681f5668d422 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:47.182283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bec67b-00ff-42fe-b74b-681f5668d422 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:47.183687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 594/1186] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:47.189590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3044f6d-efea-435e-a30e-6762e8afeeda None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:47.190007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3044f6d-efea-435e-a30e-6762e8afeeda None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:47.190146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3044f6d-efea-435e-a30e-6762e8afeeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:47.206627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b3044f6d-efea-435e-a30e-6762e8afeeda None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:47.207841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b3044f6d-efea-435e-a30e-6762e8afeeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:47.274738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 593/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:47.298048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bec67b-00ff-42fe-b74b-681f5668d422 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:47.396053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bec67b-00ff-42fe-b74b-681f5668d422 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:47.429920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a6f79-96d6-4b05-9dd5-cda992478935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:47.430690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a6f79-96d6-4b05-9dd5-cda992478935 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:47.430972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a6f79-96d6-4b05-9dd5-cda992478935 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:47.431336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a6f79-96d6-4b05-9dd5-cda992478935 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:47.441993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a6f79-96d6-4b05-9dd5-cda992478935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-983a6f79-96d6-4b05-9dd5-cda992478935 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:47.449594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:47.450692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:45:47.451670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 595/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:45:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:47.457206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3044f6d-efea-435e-a30e-6762e8afeeda None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:47.554087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3044f6d-efea-435e-a30e-6762e8afeeda None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:47.582216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:47.587951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:47.588616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:47.588899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:47.589176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:47.589913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:47.590222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:47.591619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:47.600235 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 594/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:47.606458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a6f79-96d6-4b05-9dd5-cda992478935 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:47.635306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:47.707477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a6f79-96d6-4b05-9dd5-cda992478935 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:47.740570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845c18-2657-41d9-a122-b1318ec18460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:47.741444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845c18-2657-41d9-a122-b1318ec18460 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:47.741728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845c18-2657-41d9-a122-b1318ec18460 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:47.742005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845c18-2657-41d9-a122-b1318ec18460 None admin] PATH_INFO: `/v3/users/b410464de0ea4d89ab7cd1ad447b9552` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:47.743191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad845c18-2657-41d9-a122-b1318ec18460 None admin] RBAC: Authorizing `identity:get_user(user_id=b410464de0ea4d89ab7cd1ad447b9552)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:47.744638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad845c18-2657-41d9-a122-b1318ec18460 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:47.746273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 596/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:47 2022] GET /identity/v3/users/b410464de0ea4d89ab7cd1ad447b9552 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:47.752578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:47.850263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c13ff5-cd93-472f-bd54-d56b372c2015 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:47.882626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc9566-0232-4a92-a0dc-a48c539f5e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:47.883495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9566-0232-4a92-a0dc-a48c539f5e30 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:47.883776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9566-0232-4a92-a0dc-a48c539f5e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:47.884048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9566-0232-4a92-a0dc-a48c539f5e30 None admin] PATH_INFO: `/v3/projects/4457a4c964a44aa19674aa94d82da9f2` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:47.885209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc9566-0232-4a92-a0dc-a48c539f5e30 None admin] RBAC: Authorizing `identity:get_project(project_id=4457a4c964a44aa19674aa94d82da9f2)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:47.886620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc9566-0232-4a92-a0dc-a48c539f5e30 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:47.888331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 595/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:47 2022] GET /identity/v3/projects/4457a4c964a44aa19674aa94d82da9f2 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:47.894842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845c18-2657-41d9-a122-b1318ec18460 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:47.988405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:47.994004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845c18-2657-41d9-a122-b1318ec18460 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:48.028205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511ec21-da86-47af-bb3d-870b4f3e0fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:48.029084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511ec21-da86-47af-bb3d-870b4f3e0fde None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:48.029363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511ec21-da86-47af-bb3d-870b4f3e0fde None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:48.029665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511ec21-da86-47af-bb3d-870b4f3e0fde None admin] PATH_INFO: `/v3/projects/4457a4c964a44aa19674aa94d82da9f2/users/b410464de0ea4d89ab7cd1ad447b9552/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:48.031634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511ec21-da86-47af-bb3d-870b4f3e0fde None admin] RBAC: Authorizing `identity:create_grant(project_id=4457a4c964a44aa19674aa94d82da9f2, user_id=b410464de0ea4d89ab7cd1ad447b9552, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:48.033066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511ec21-da86-47af-bb3d-870b4f3e0fde None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:48.094933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 597/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:45:47 2022] PUT /identity/v3/projects/4457a4c964a44aa19674aa94d82da9f2/users/b410464de0ea4d89ab7cd1ad447b9552/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:45:50.126163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb00c-707e-4633-aee2-99acc8a66c03 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:50.126716 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb00c-707e-4633-aee2-99acc8a66c03 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:50.126716 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb00c-707e-4633-aee2-99acc8a66c03 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:50.128128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 596/1193] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:45:50.133213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dac3d-cbf5-4a67-86a5-f46c48d052af None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:50.133598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dac3d-cbf5-4a67-86a5-f46c48d052af None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:50.133782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dac3d-cbf5-4a67-86a5-f46c48d052af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:50.150644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d99dac3d-cbf5-4a67-86a5-f46c48d052af None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:45:50.151864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d99dac3d-cbf5-4a67-86a5-f46c48d052af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:45:50.219355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 598/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:45:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:45:50.243386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bb00c-707e-4633-aee2-99acc8a66c03 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:50.302666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:50.343812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bb00c-707e-4633-aee2-99acc8a66c03 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:50.375327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e48e5-a0ff-4bc9-9b44-31a187e896f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:50.376096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e48e5-a0ff-4bc9-9b44-31a187e896f0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:50.376432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e48e5-a0ff-4bc9-9b44-31a187e896f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:50.376651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e48e5-a0ff-4bc9-9b44-31a187e896f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:50.386298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e48e5-a0ff-4bc9-9b44-31a187e896f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-fa9e48e5-a0ff-4bc9-9b44-31a187e896f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:45:50.394262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:45:50.395426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:45:50.396278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 597/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:45:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:45:50.401637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dac3d-cbf5-4a67-86a5-f46c48d052af None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:50.482834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:50.496885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dac3d-cbf5-4a67-86a5-f46c48d052af None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:50.534899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:50.535598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:50.535881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:50.536160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:50.536909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:50.537224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:50.538589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:50.547213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 599/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:45:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:50.553018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e48e5-a0ff-4bc9-9b44-31a187e896f0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:50.650016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e48e5-a0ff-4bc9-9b44-31a187e896f0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:50.669017 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:50.682793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec8c5b-a0cc-4b63-998c-3359e4cac54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:50.683681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec8c5b-a0cc-4b63-998c-3359e4cac54d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:50.683959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec8c5b-a0cc-4b63-998c-3359e4cac54d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:50.684225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec8c5b-a0cc-4b63-998c-3359e4cac54d None admin] PATH_INFO: `/v3/users/b410464de0ea4d89ab7cd1ad447b9552` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:50.685397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec8c5b-a0cc-4b63-998c-3359e4cac54d None admin] RBAC: Authorizing `identity:get_user(user_id=b410464de0ea4d89ab7cd1ad447b9552)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:50.687436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec8c5b-a0cc-4b63-998c-3359e4cac54d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:50.689071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 598/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 1 03:45:50 2022] GET /identity/v3/users/b410464de0ea4d89ab7cd1ad447b9552 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:50.695469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:50.793391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f592dd3-c887-4e9b-814b-dcc8fccc5732 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:50.827080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aced5b-8fbe-4a86-ad24-62613a242735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:50.827750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aced5b-8fbe-4a86-ad24-62613a242735 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:50.828032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aced5b-8fbe-4a86-ad24-62613a242735 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:50.828308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aced5b-8fbe-4a86-ad24-62613a242735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:50.845505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:50.895273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aced5b-8fbe-4a86-ad24-62613a242735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:50.896719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aced5b-8fbe-4a86-ad24-62613a242735 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:50.996806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 600/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:45:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:45:51.003452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec8c5b-a0cc-4b63-998c-3359e4cac54d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:51.100345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec8c5b-a0cc-4b63-998c-3359e4cac54d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:51.133394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb3960-70e3-46cd-84f8-ca7e0063bc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:51.134240 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb3960-70e3-46cd-84f8-ca7e0063bc18 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:51.134522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb3960-70e3-46cd-84f8-ca7e0063bc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:51.134804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb3960-70e3-46cd-84f8-ca7e0063bc18 None admin] PATH_INFO: `/v3/projects/4457a4c964a44aa19674aa94d82da9f2` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:51.136000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb3960-70e3-46cd-84f8-ca7e0063bc18 None admin] RBAC: Authorizing `identity:get_project(project_id=4457a4c964a44aa19674aa94d82da9f2)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:51.137426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb3960-70e3-46cd-84f8-ca7e0063bc18 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:51.139144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 599/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 1 03:45:51 2022] GET /identity/v3/projects/4457a4c964a44aa19674aa94d82da9f2 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:45:51.145281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aced5b-8fbe-4a86-ad24-62613a242735 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:45:51.193597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:45:51.244587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aced5b-8fbe-4a86-ad24-62613a242735 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:45:51.277388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04a395-128e-4a78-82a8-3de256eb6ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:45:51.278031 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04a395-128e-4a78-82a8-3de256eb6ead None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:45:51.278310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04a395-128e-4a78-82a8-3de256eb6ead None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:45:51.278583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04a395-128e-4a78-82a8-3de256eb6ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:45:51.279935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04a395-128e-4a78-82a8-3de256eb6ead None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=4457a4c964a44aa19674aa94d82da9f2, user.id=b410464de0ea4d89ab7cd1ad447b9552) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:45:51.280250 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04a395-128e-4a78-82a8-3de256eb6ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:45:51.281659 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04a395-128e-4a78-82a8-3de256eb6ead None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:45:51.287514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 601/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:45:51 2022] GET /identity/v3/role_assignments?user.id=b410464de0ea4d89ab7cd1ad447b9552&scope.project.id=4457a4c964a44aa19674aa94d82da9f2&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:46:01.129163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=72 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:46:01.292463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=44 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:11.180855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa77e5-5b42-4a13-8dea-96f8c3d551d3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:11.181438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa77e5-5b42-4a13-8dea-96f8c3d551d3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:11.181438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa77e5-5b42-4a13-8dea-96f8c3d551d3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:11.182773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 600/1201] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:11.188111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513ec76-39fd-4b0b-a9dc-ecd01c30457c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:11.191120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513ec76-39fd-4b0b-a9dc-ecd01c30457c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:11.191120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513ec76-39fd-4b0b-a9dc-ecd01c30457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:11.206361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4513ec76-39fd-4b0b-a9dc-ecd01c30457c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:11.207722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4513ec76-39fd-4b0b-a9dc-ecd01c30457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:11.276580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 602/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:11.303066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa77e5-5b42-4a13-8dea-96f8c3d551d3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:11.404857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa77e5-5b42-4a13-8dea-96f8c3d551d3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:11.439266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:11.440043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:11.440324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:11.440601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:11.450638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:11.452187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:11.470769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:11.471944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:11.472923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:11.472923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:11.472923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:11.472923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:11.472923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 601/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 03:47:11.478288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513ec76-39fd-4b0b-a9dc-ecd01c30457c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:11.577571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513ec76-39fd-4b0b-a9dc-ecd01c30457c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:11.606323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:11.612054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24099503-2eda-4d7c-8099-3a36d45159b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:11.612714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24099503-2eda-4d7c-8099-3a36d45159b4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:11.613002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24099503-2eda-4d7c-8099-3a36d45159b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:11.613282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24099503-2eda-4d7c-8099-3a36d45159b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:11.614003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24099503-2eda-4d7c-8099-3a36d45159b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:11.614313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24099503-2eda-4d7c-8099-3a36d45159b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:11.615701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24099503-2eda-4d7c-8099-3a36d45159b4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:11.626773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 603/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:11.632769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:11.687743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:11.738263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58f9a2-2ded-4b92-978e-8289d8f6f07c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:11.771190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e0ab4-291d-411a-ad79-f4720a36e334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:11.771885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e0ab4-291d-411a-ad79-f4720a36e334 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:11.772170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e0ab4-291d-411a-ad79-f4720a36e334 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:11.772445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e0ab4-291d-411a-ad79-f4720a36e334 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:11.773152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e0ab4-291d-411a-ad79-f4720a36e334 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:11.774579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e0ab4-291d-411a-ad79-f4720a36e334 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:11.982327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 602/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 1 03:47:11 2022] POST /identity/v3/users => generated 251 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:47:14.053197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ad48-fc73-492e-a832-209c12f2e983 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:14.054784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ad48-fc73-492e-a832-209c12f2e983 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:14.054784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ad48-fc73-492e-a832-209c12f2e983 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:14.055405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 604/1206] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:14.060809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa80ba-a54b-4942-9402-7fe5e2c16dc4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:14.061267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa80ba-a54b-4942-9402-7fe5e2c16dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:14.061541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa80ba-a54b-4942-9402-7fe5e2c16dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:14.079832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-32aa80ba-a54b-4942-9402-7fe5e2c16dc4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:14.084159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-32aa80ba-a54b-4942-9402-7fe5e2c16dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:14.149496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 603/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:47:14.173716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ad48-fc73-492e-a832-209c12f2e983 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:14.275977 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ad48-fc73-492e-a832-209c12f2e983 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:14.316159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60348eb-7db6-45ac-b1be-6dc5fcdd4ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:14.316876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60348eb-7db6-45ac-b1be-6dc5fcdd4ffa None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:14.317157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60348eb-7db6-45ac-b1be-6dc5fcdd4ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:14.317432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60348eb-7db6-45ac-b1be-6dc5fcdd4ffa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:14.327926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60348eb-7db6-45ac-b1be-6dc5fcdd4ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e60348eb-7db6-45ac-b1be-6dc5fcdd4ffa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:47:14.336545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:14.337728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:14.338819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 605/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:47:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:14.343982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa80ba-a54b-4942-9402-7fe5e2c16dc4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:14.385163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:14.449021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa80ba-a54b-4942-9402-7fe5e2c16dc4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:14.483349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:14.492591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:14.492591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:14.492591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:14.492591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:14.492591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:14.492591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:14.495917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 604/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:47:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:14.502429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60348eb-7db6-45ac-b1be-6dc5fcdd4ffa None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:14.559072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:14.611448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60348eb-7db6-45ac-b1be-6dc5fcdd4ffa None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:14.645289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:14.646079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:14.646364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:14.646638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:14.657932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:14.659407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:14.677504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:14.678674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:14.680208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:14.680208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:14.680208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:14.680208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:14.680208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 606/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:14.685521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:14.793960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:14.796853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91626b16-aeec-4910-8a74-c52faf2188d6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:14.897217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:14.897217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:14.897217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:14.897217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:14.897875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:14.897875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:14.897875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:14.897875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 605/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:14.898415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:14.959407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a536b-ef96-47bb-9c41-51a877d3f934 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:14.960979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:14.995482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:14.996279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:14.996562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:14.996837 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:15.004037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:15.005515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:47:15.020005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:15.021310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:15.022642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 607/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:15.027013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:15.135561 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c2852-1c03-46e9-a107-334d51b20ac2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:15.174409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:15.175041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:15.175384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:15.175661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:15.176554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:15.176900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:15.178368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:15.199272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 606/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 1 03:47:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:15.205489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:15.312109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f6c04-7e7c-4644-9695-02f555257150 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:15.347256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e870f-6731-435d-8df7-4b41756eece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:15.347906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e870f-6731-435d-8df7-4b41756eece0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:15.348234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e870f-6731-435d-8df7-4b41756eece0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:15.348504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e870f-6731-435d-8df7-4b41756eece0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:15.380147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:15.418033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e870f-6731-435d-8df7-4b41756eece0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:15.419504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e870f-6731-435d-8df7-4b41756eece0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:15.518137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 608/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:47:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:47:15.524288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:15.549307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:15.632339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd28a6-03b0-420c-9cc4-08a023e3a1a8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:15.667267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:15.668053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:15.668334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:15.668607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:15.678929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:15.681247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:15.700985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:15.702143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:15.703274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:15.703274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:15.703274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:15.703274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:15.703274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 607/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:15.708400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e870f-6731-435d-8df7-4b41756eece0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:15.770485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:15.808840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e870f-6731-435d-8df7-4b41756eece0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:15.841960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:15.842616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:15.842900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:15.843175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:15.843920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:15.844229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:15.845634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:15.857218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 609/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:15.864360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:15.937532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:15.968521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd99a4-cda2-445b-8b71-a882a1029765 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:16.002526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:16.003376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:16.003662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:16.003939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:16.010978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:16.012449 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:47:16.032527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:16.033698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:16.034773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 608/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:16.035494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:16.134902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a284f1-1ccc-4b68-bdae-ff0d4003e245 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:16.158571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:16.170457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e748786-d0f4-46b6-b4c2-9469626982c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:16.171118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748786-d0f4-46b6-b4c2-9469626982c3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:16.171434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748786-d0f4-46b6-b4c2-9469626982c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:16.171712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748786-d0f4-46b6-b4c2-9469626982c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:16.173053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e748786-d0f4-46b6-b4c2-9469626982c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:16.173327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e748786-d0f4-46b6-b4c2-9469626982c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:16.174760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e748786-d0f4-46b6-b4c2-9469626982c3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:16.186683 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 610/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:16.193423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:16.302244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de617e1-afd1-4b28-a2ce-dbd05d45564b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:16.330717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:16.336445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c886f-e4f6-469c-a955-466d539be4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:16.337118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c886f-e4f6-469c-a955-466d539be4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:16.337392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c886f-e4f6-469c-a955-466d539be4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:16.337662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c886f-e4f6-469c-a955-466d539be4a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:16.338475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c886f-e4f6-469c-a955-466d539be4a2 None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=f408d50bfc3a4366aabb1cdf86da7935) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:16.338780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c886f-e4f6-469c-a955-466d539be4a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:16.340220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c886f-e4f6-469c-a955-466d539be4a2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:16.345979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 609/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:47:16 2022] GET /identity/v3/role_assignments?user.id=f408d50bfc3a4366aabb1cdf86da7935&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:18.362012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa6973-9929-49f0-8add-20dc787457f5 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:18.362582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa6973-9929-49f0-8add-20dc787457f5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:18.362582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa6973-9929-49f0-8add-20dc787457f5 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:18.363957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 611/1220] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:18.369254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3ceff-17f2-4ed0-8e09-74687b846c57 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:18.369714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3ceff-17f2-4ed0-8e09-74687b846c57 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:18.369821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3ceff-17f2-4ed0-8e09-74687b846c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:18.387817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3bd3ceff-17f2-4ed0-8e09-74687b846c57 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:18.389068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3bd3ceff-17f2-4ed0-8e09-74687b846c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:18.458608 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 610/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:18.482625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa6973-9929-49f0-8add-20dc787457f5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:18.591004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa6973-9929-49f0-8add-20dc787457f5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:18.627781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4448f-0b4d-4062-ac4f-a4072e42e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:18.628506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4448f-0b4d-4062-ac4f-a4072e42e09b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:18.628777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4448f-0b4d-4062-ac4f-a4072e42e09b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:18.629062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4448f-0b4d-4062-ac4f-a4072e42e09b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:18.639309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4448f-0b4d-4062-ac4f-a4072e42e09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-84e4448f-0b4d-4062-ac4f-a4072e42e09b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:47:18.646861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:18.654376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:18.655560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 612/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:47:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:18.656527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3ceff-17f2-4ed0-8e09-74687b846c57 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:18.756391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3ceff-17f2-4ed0-8e09-74687b846c57 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:18.779956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:18.791108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:18.791808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:18.792094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:18.792371 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:18.793145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:18.793425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:18.794850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:18.803754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 611/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:47:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:18.809967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4448f-0b4d-4062-ac4f-a4072e42e09b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:18.855801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:18.914968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4448f-0b4d-4062-ac4f-a4072e42e09b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:18.949582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:18.950372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:18.950654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:18.950929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:18.962230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:18.963915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:18.981130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:18.982232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:18.983375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:18.983375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:18.983375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:18.983375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:18.983375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 613/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:18.988924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:19.095889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac4da2-e3b9-461f-8096-8501c47f1a1d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:19.131461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:19.132119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:19.132407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:19.132683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:19.133403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:19.133711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:19.135069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:19.147161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 612/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:19.153755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:19.253926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:19.260681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126bdb1-6de9-4275-a06f-6835d6624632 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:19.307055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:19.307876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:19.308156 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:19.308427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:19.315789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:19.317220 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:47:19.331777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:19.333877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:19.335888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 614/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:19.339294 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:19.386740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:19.456012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b3734-b866-40c6-a2fb-d29b321e9b95 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:19.492766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:19.493439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:19.493717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:19.493988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:19.494828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:19.495135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:19.496615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:19.517653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 613/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 1 03:47:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:19.523600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:19.625415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9129561-90c5-4657-8b51-33470eb2e98c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:19.659988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:19.660768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:19.661044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:19.661317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:19.671492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:19.676385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:19.692293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:19.693425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:19.694689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:19.694689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:19.694689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:19.694689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:19.694689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 615/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:19.699403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:19.800561 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaec45-15a5-4c09-b6dd-b8f83d0a8020 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:19.802836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:19.834705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:19.835404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:19.835688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:19.835960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:19.836672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:19.836976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:19.838331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:19.850172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 614/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:19.856160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:19.865888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:19.956742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf00003-b8e2-49b5-bc7b-a26688382f7a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:19.990367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:19.991206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:19.991525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:19.991804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:19.998354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:19.999903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:47:20.014341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:20.015522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:20.016745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 616/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:47:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:20.022011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:20.124438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60674ec2-c28e-4de3-8e89-9e34a621fddb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:20.158754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bbb35-6d1d-47b1-a949-7129d21ab149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:20.159456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbb35-6d1d-47b1-a949-7129d21ab149 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:20.159734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbb35-6d1d-47b1-a949-7129d21ab149 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:20.160007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbb35-6d1d-47b1-a949-7129d21ab149 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:20.160788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bbb35-6d1d-47b1-a949-7129d21ab149 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:20.161098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bbb35-6d1d-47b1-a949-7129d21ab149 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:20.162511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bbb35-6d1d-47b1-a949-7129d21ab149 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:20.174967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 615/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:20.181363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:20.246952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:20.280846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a359a76-a8d1-4d42-b4d0-3a3c7a68bb8b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:20.315481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37241a-8da8-4ea7-8173-dba447528412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:20.316367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37241a-8da8-4ea7-8173-dba447528412 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:20.316655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37241a-8da8-4ea7-8173-dba447528412 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:20.316934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37241a-8da8-4ea7-8173-dba447528412 None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/f408d50bfc3a4366aabb1cdf86da7935/roles/d31dcaade43b4d03a991cb76b4c830af` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:20.336692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37241a-8da8-4ea7-8173-dba447528412 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=f408d50bfc3a4366aabb1cdf86da7935, role_id=d31dcaade43b4d03a991cb76b4c830af)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:20.338150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37241a-8da8-4ea7-8173-dba447528412 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:20.476182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 617/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:47:20 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/f408d50bfc3a4366aabb1cdf86da7935/roles/d31dcaade43b4d03a991cb76b4c830af => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:47:22.524348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cdbfb-3ffd-4c4a-aaf5-5ef17e66867b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:22.524948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cdbfb-3ffd-4c4a-aaf5-5ef17e66867b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:22.524948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cdbfb-3ffd-4c4a-aaf5-5ef17e66867b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:22.526287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 616/1233] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:22.531598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e82136-4b01-4290-9bc8-c0a43d2d77ef None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:22.532566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e82136-4b01-4290-9bc8-c0a43d2d77ef None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:22.532658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e82136-4b01-4290-9bc8-c0a43d2d77ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:22.551078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e1e82136-4b01-4290-9bc8-c0a43d2d77ef None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:22.552336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e1e82136-4b01-4290-9bc8-c0a43d2d77ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:22.624496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 618/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:22.648500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cdbfb-3ffd-4c4a-aaf5-5ef17e66867b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:22.674944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:22.760265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cdbfb-3ffd-4c4a-aaf5-5ef17e66867b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:22.796665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5febcc-9cc1-4ebf-97a7-14beecb05fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:22.797887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5febcc-9cc1-4ebf-97a7-14beecb05fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:22.798175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5febcc-9cc1-4ebf-97a7-14beecb05fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:22.798454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5febcc-9cc1-4ebf-97a7-14beecb05fb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:22.809488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5febcc-9cc1-4ebf-97a7-14beecb05fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5e5febcc-9cc1-4ebf-97a7-14beecb05fb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:47:22.818010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:22.819020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:22.819913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 617/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 1 03:47:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:22.825513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e82136-4b01-4290-9bc8-c0a43d2d77ef None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:22.873952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:22.929076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e82136-4b01-4290-9bc8-c0a43d2d77ef None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:22.963720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:22.964392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:22.964679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:22.964959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:22.965733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:22.966044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:22.967427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:22.975672 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 619/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 1 03:47:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:22.981390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5febcc-9cc1-4ebf-97a7-14beecb05fb1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:23.076236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:23.092305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5febcc-9cc1-4ebf-97a7-14beecb05fb1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:23.128487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:23.129275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:23.129615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:23.129877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:23.140089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:23.141548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:23.159488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:23.160512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:23.161441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:23.161441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:23.161441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:23.161441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:23.161441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 618/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:23.166746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:23.273587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f1e08-ef5d-4a6f-9b32-533ce81c6b9a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:23.283294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:23.309424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:23.310242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:23.310528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:23.310813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:23.311585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:23.311897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:23.313317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:23.325998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 620/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:23.332030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:23.439558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb799a8-2970-42f8-a396-327e5d787095 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:23.475543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:23.476331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:23.476607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:23.476885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:23.479782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:23.484540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:23.486037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:47:23.500386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:23.501522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:23.502631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 619/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:23.507606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:23.621240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454103b-8fab-4682-9e01-38cb2f65f06c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:23.659016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:23.659734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:23.660023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:23.660302 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:23.661149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:23.661464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:23.662906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:23.682192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 621/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 1 03:47:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:23.688183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:23.792890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b192eb-b808-4442-bc63-99d0c6e77090 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:23.827580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c468e0-b38e-438c-9e07-178643fb23ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:23.828223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c468e0-b38e-438c-9e07-178643fb23ee None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:23.828500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c468e0-b38e-438c-9e07-178643fb23ee None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:23.829330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c468e0-b38e-438c-9e07-178643fb23ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:23.892580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:23.900589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c468e0-b38e-438c-9e07-178643fb23ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:23.902888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c468e0-b38e-438c-9e07-178643fb23ee None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:24.003852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 620/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:47:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:47:24.010280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:24.037823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:24.107883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866202-3f98-45bc-9757-1dc8ad2294b9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:24.142977 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:24.143726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:24.144030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:24.144301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:24.154149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:24.155598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:24.174074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:24.175251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:24.176223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:24.176223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:24.176223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:24.176223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:24.176223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 622/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:24.181443 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c468e0-b38e-438c-9e07-178643fb23ee None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:24.275723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:24.281239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c468e0-b38e-438c-9e07-178643fb23ee None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:24.315123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:24.315804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:24.316083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:24.316354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:24.317064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:24.317368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:24.318712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:24.329779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 621/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:24.335735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:24.415649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:24.438662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ba4bb-ce7f-41bd-b4a3-b28701da9249 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:24.472403 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:24.473186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:24.473472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:24.473750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:24.480448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:24.481881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:47:24.495830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:24.496860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:24.497756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 623/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:24.503742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:24.615175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57696f-4ed4-424d-b89c-7ce527969582 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:24.649221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639879b-3d4c-4022-94eb-9ac06cb1be72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:24.649878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639879b-3d4c-4022-94eb-9ac06cb1be72 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:24.650161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639879b-3d4c-4022-94eb-9ac06cb1be72 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:24.650440 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639879b-3d4c-4022-94eb-9ac06cb1be72 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:24.651255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639879b-3d4c-4022-94eb-9ac06cb1be72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:24.651565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639879b-3d4c-4022-94eb-9ac06cb1be72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:24.652995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639879b-3d4c-4022-94eb-9ac06cb1be72 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:24.665408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 622/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:47:24.671898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:24.782751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37509911-309e-4308-abd9-83ee138038e0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:24.818306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1193f23-5d6f-4957-84a9-6510bdc3155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:24.818952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1193f23-5d6f-4957-84a9-6510bdc3155c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:24.819258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1193f23-5d6f-4957-84a9-6510bdc3155c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:24.819540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1193f23-5d6f-4957-84a9-6510bdc3155c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:24.820389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1193f23-5d6f-4957-84a9-6510bdc3155c None admin] RBAC: Adding query filter params (role.id=d31dcaade43b4d03a991cb76b4c830af, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=f408d50bfc3a4366aabb1cdf86da7935) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:24.820699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1193f23-5d6f-4957-84a9-6510bdc3155c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:24.822133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1193f23-5d6f-4957-84a9-6510bdc3155c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:24.827856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 624/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:47:24 2022] GET /identity/v3/role_assignments?user.id=f408d50bfc3a4366aabb1cdf86da7935&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=d31dcaade43b4d03a991cb76b4c830af => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:26.936015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f5334-2360-4dad-8f6d-8bc467312797 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:26.936581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f5334-2360-4dad-8f6d-8bc467312797 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:26.936581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f5334-2360-4dad-8f6d-8bc467312797 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:26.937970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 623/1247] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:26.943280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a60b-fb0b-4a2c-bea1-1bc3c9d214ff None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:26.943749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a60b-fb0b-4a2c-bea1-1bc3c9d214ff None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:26.943898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a60b-fb0b-4a2c-bea1-1bc3c9d214ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:26.949108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:26.961103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a832a60b-fb0b-4a2c-bea1-1bc3c9d214ff None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:26.962222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a832a60b-fb0b-4a2c-bea1-1bc3c9d214ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:27.030107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 625/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:27.054992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f5334-2360-4dad-8f6d-8bc467312797 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:27.079136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:27.164877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f5334-2360-4dad-8f6d-8bc467312797 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:27.200952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdb5e9-db1a-4cec-9e84-6bf3671f5a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:27.201776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdb5e9-db1a-4cec-9e84-6bf3671f5a44 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:27.202057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdb5e9-db1a-4cec-9e84-6bf3671f5a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:27.202327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdb5e9-db1a-4cec-9e84-6bf3671f5a44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:27.214405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdb5e9-db1a-4cec-9e84-6bf3671f5a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-aabdb5e9-db1a-4cec-9e84-6bf3671f5a44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:47:27.229356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:27.230465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:27.231685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 624/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:47:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:27.237725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832a60b-fb0b-4a2c-bea1-1bc3c9d214ff None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:27.336749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832a60b-fb0b-4a2c-bea1-1bc3c9d214ff None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:27.367250 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:27.372565 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:27.373235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:27.373517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:27.373792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:27.374538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:27.374847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:27.376780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:27.386635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 626/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:27.391089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdb5e9-db1a-4cec-9e84-6bf3671f5a44 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:27.473453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:27.497011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdb5e9-db1a-4cec-9e84-6bf3671f5a44 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:27.531303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:27.532605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:27.532885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:27.533169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:27.544355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:27.545804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:27.564195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:27.565350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:27.566667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:27.566667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:27.566667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:27.566667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:27.566667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 625/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:27.571536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:27.674884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a144b3-f419-4f10-9367-cf5aa08e139f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:27.710601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:27.711295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:27.711580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:27.711953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:27.712585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:27.713255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:27.714280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:27.727087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 627/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:27.732895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:27.839543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c67a7-0404-415d-b991-2767cdb3d3c3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:27.869262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:27.877669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:27.878496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:27.878778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:27.879052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:27.888566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:27.890043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:47:27.906188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:27.907864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:27.908881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 626/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:27.914254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:27.972419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:28.021984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8d69d-bbef-42f7-a18a-7b63026fe9f7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:28.056882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:28.057602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:28.057886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:28.058165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:28.059005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:28.059348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:28.060780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:28.078034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 628/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 1 03:47:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:28.084319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:28.185105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1be07-1d82-4e3e-9267-586eb530f627 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:28.221244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c88d89-dca8-440c-b9f9-8bd8c7d2190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:28.221886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c88d89-dca8-440c-b9f9-8bd8c7d2190e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:28.222168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c88d89-dca8-440c-b9f9-8bd8c7d2190e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:28.222480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c88d89-dca8-440c-b9f9-8bd8c7d2190e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:28.277378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:28.289976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c88d89-dca8-440c-b9f9-8bd8c7d2190e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:28.291452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c88d89-dca8-440c-b9f9-8bd8c7d2190e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:28.387327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 627/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:47:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:47:28.393831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:28.497237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e94667-fdc0-4647-823c-c22ff0b8c179 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:28.511905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:28.532688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:28.533412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:28.533695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:28.533972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:28.544267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:28.545723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:28.564513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:28.565632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:28.571347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:28.571347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:28.571347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:28.571347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:28.571347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 629/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:28.575379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c88d89-dca8-440c-b9f9-8bd8c7d2190e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:28.665465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:28.677028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c88d89-dca8-440c-b9f9-8bd8c7d2190e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:28.711563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:28.712228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:28.712509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:28.712782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:28.713540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:28.713863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:28.715204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:28.726897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 628/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 03:47:28.732668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:28.832079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a9ed1-8929-4556-bd6b-d8dab3cc93bb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:28.866291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:28.867072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:28.867382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:28.867653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:28.874582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:28.881838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:47:28.889933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:28.891523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:28.893196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 630/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:47:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:28.898030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:29.001244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084fcd5-6c89-447a-a76d-4cf835c441ae None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:29.036399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffa5c7-4216-4bdc-b49d-992702dc5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:29.037059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffa5c7-4216-4bdc-b49d-992702dc5723 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:29.037345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffa5c7-4216-4bdc-b49d-992702dc5723 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:29.037617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffa5c7-4216-4bdc-b49d-992702dc5723 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:29.038400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffa5c7-4216-4bdc-b49d-992702dc5723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:29.038709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffa5c7-4216-4bdc-b49d-992702dc5723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:29.040166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffa5c7-4216-4bdc-b49d-992702dc5723 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:29.052523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 629/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:47:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:29.058931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:29.094335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:29.171036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba2c61-7858-4086-b051-f70101c0e6c3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:29.208710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b5693-df2e-4185-8c8c-cc296c2700fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:29.209361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b5693-df2e-4185-8c8c-cc296c2700fa None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:29.209640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b5693-df2e-4185-8c8c-cc296c2700fa None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:29.209913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b5693-df2e-4185-8c8c-cc296c2700fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:29.210731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b5693-df2e-4185-8c8c-cc296c2700fa None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=f408d50bfc3a4366aabb1cdf86da7935) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:29.211132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b5693-df2e-4185-8c8c-cc296c2700fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:29.212687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b5693-df2e-4185-8c8c-cc296c2700fa None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:29.218270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 631/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:47:29 2022] GET /identity/v3/role_assignments?user.id=f408d50bfc3a4366aabb1cdf86da7935&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:31.277972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0f19b-e4cc-4c6a-ad74-0639aafb364d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:31.278544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0f19b-e4cc-4c6a-ad74-0639aafb364d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:31.278544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0f19b-e4cc-4c6a-ad74-0639aafb364d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:31.279925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 630/1261] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:31.286214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7c50c-fb33-4abf-87d4-e3717341bb8c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:31.286733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7c50c-fb33-4abf-87d4-e3717341bb8c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:31.286841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7c50c-fb33-4abf-87d4-e3717341bb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:31.304222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-09f7c50c-fb33-4abf-87d4-e3717341bb8c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:31.305693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-09f7c50c-fb33-4abf-87d4-e3717341bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:31.379876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 632/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:31.403106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0f19b-e4cc-4c6a-ad74-0639aafb364d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:31.414609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:31.515203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0f19b-e4cc-4c6a-ad74-0639aafb364d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:31.552147 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6bb47-14c2-4318-b34e-9bbe4b9ea073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:31.552365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6bb47-14c2-4318-b34e-9bbe4b9ea073 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:31.552574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6bb47-14c2-4318-b34e-9bbe4b9ea073 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:31.552844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6bb47-14c2-4318-b34e-9bbe4b9ea073 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:31.563325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6bb47-14c2-4318-b34e-9bbe4b9ea073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-64d6bb47-14c2-4318-b34e-9bbe4b9ea073 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:47:31.571171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:31.572417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:31.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 631/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:31.578738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7c50c-fb33-4abf-87d4-e3717341bb8c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:31.598198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:31.681200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7c50c-fb33-4abf-87d4-e3717341bb8c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:31.716725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:31.717395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:31.717679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:31.717958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:31.718716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:31.719058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:31.720590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:31.729027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 633/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:31.735474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6bb47-14c2-4318-b34e-9bbe4b9ea073 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:31.807214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:31.841842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6bb47-14c2-4318-b34e-9bbe4b9ea073 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:31.876041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:31.876830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:31.877113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:31.877397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:31.888263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:31.889767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:31.907031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:31.908981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:31.910072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:31.910072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:31.910072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:31.910072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:31.910072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 632/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:31.914185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:31.997913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:32.028233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3c699-8dde-47a1-8328-0901dd462497 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:32.063044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:32.063651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:32.063931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:32.064205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:32.064980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:32.065281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:32.066682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:32.078054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 634/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:32.084000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:32.193293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e3ef8-8ec4-477b-acbe-2a590271806a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:32.215785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:32.231834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:32.232665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:32.232955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:32.233231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:32.242712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:32.244242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:47:32.260789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:32.262131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:32.263286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 633/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:32.269051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:32.373412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0762d-506d-41a9-b7bb-9a3e157339fc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:32.408363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:32.409034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:32.409317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:32.409597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:32.410427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:32.410740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:32.412407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:32.421958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:32.430397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 635/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 1 03:47:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:32.436424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:32.536582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e625f3-c781-4db3-ae34-d14d3d0e3dd3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:32.570351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:32.571132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:32.571484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:32.571715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:32.582674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:32.584140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:32.601453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:32.602649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:32.603926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:32.603926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:32.603926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:32.603926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:32.603926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 634/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:32.608521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:32.716236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4737b-a175-4f65-8741-f33e8368afd2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:32.749968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:32.751248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:32.751539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:32.751812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:32.752531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:32.752837 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:32.754189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:32.765324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 636/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:32.771022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:32.803979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:32.872188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cf720-2b1a-4aca-af1b-f8f54306a1b5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:32.907843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:32.908681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:32.908959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:32.909230 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:32.916235 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:32.917651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:47:32.931828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:32.933085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:32.934174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 635/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:32.939692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:32.997943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:33.038311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fd5e2-3b13-4576-b8f9-cc46a8311c8d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:33.070900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97229ec7-f03d-436c-a2d7-052ca7bdd08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:33.071583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97229ec7-f03d-436c-a2d7-052ca7bdd08f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:33.071862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97229ec7-f03d-436c-a2d7-052ca7bdd08f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:33.072140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97229ec7-f03d-436c-a2d7-052ca7bdd08f None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:33.072923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97229ec7-f03d-436c-a2d7-052ca7bdd08f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:33.073230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97229ec7-f03d-436c-a2d7-052ca7bdd08f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:33.074670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97229ec7-f03d-436c-a2d7-052ca7bdd08f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:33.087156 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 637/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:33.093125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:33.192566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:33.201438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8a588-4909-4ec1-9036-1a5bc55915fb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:33.236131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6cf79-4c6f-489c-a9c4-49a76f6f7c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:33.237041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6cf79-4c6f-489c-a9c4-49a76f6f7c7d None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:33.237325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6cf79-4c6f-489c-a9c4-49a76f6f7c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:33.237604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6cf79-4c6f-489c-a9c4-49a76f6f7c7d None admin] PATH_INFO: `/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/f408d50bfc3a4366aabb1cdf86da7935/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:33.239858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6cf79-4c6f-489c-a9c4-49a76f6f7c7d None admin] RBAC: Authorizing `identity:create_grant(project_id=6bcddee341bf4069b0154f2eceb30cf3, user_id=f408d50bfc3a4366aabb1cdf86da7935, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:33.241297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6cf79-4c6f-489c-a9c4-49a76f6f7c7d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:33.362027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 636/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 1 03:47:33 2022] PUT /identity/v3/projects/6bcddee341bf4069b0154f2eceb30cf3/users/f408d50bfc3a4366aabb1cdf86da7935/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 03:47:35.421857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eb42c-b5cb-4413-9119-1630012656f1 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:35.423036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eb42c-b5cb-4413-9119-1630012656f1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:35.423036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eb42c-b5cb-4413-9119-1630012656f1 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:35.424399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 638/1274] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:35.430154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b27c-9d6b-44b2-938e-25e5c9d244c1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:35.430615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b27c-9d6b-44b2-938e-25e5c9d244c1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:35.430712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b27c-9d6b-44b2-938e-25e5c9d244c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:35.448213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f203b27c-9d6b-44b2-938e-25e5c9d244c1 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:35.449427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f203b27c-9d6b-44b2-938e-25e5c9d244c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:35.520448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 637/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:35.544796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eb42c-b5cb-4413-9119-1630012656f1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:35.648536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eb42c-b5cb-4413-9119-1630012656f1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:35.666439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:35.682961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197d7e0-ac4b-4ebd-a3f2-a759b0875d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:35.683707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197d7e0-ac4b-4ebd-a3f2-a759b0875d8b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:35.683989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197d7e0-ac4b-4ebd-a3f2-a759b0875d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:35.684264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197d7e0-ac4b-4ebd-a3f2-a759b0875d8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:35.695366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197d7e0-ac4b-4ebd-a3f2-a759b0875d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e197d7e0-ac4b-4ebd-a3f2-a759b0875d8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 03:47:35.702988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:35.704682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:35.705736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 639/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 1 03:47:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:35.710795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203b27c-9d6b-44b2-938e-25e5c9d244c1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:35.795527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:35.814135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203b27c-9d6b-44b2-938e-25e5c9d244c1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:35.848900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:35.849590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:35.849867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:35.850134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:35.850873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:35.851178 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:35.852559 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:35.860816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 638/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 1 03:47:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:35.866804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197d7e0-ac4b-4ebd-a3f2-a759b0875d8b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:35.981548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197d7e0-ac4b-4ebd-a3f2-a759b0875d8b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:36.016492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:36.017340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:36.017639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:36.017906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:36.031942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:36.033418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:36.051063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:36.052230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:36.053355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:36.053355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:36.053355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:36.053355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:36.053355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 640/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:36.058725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:36.158722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012c612-d234-4fde-a089-78abc0fdc819 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:36.191349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:36.192050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:36.192335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:36.192610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:36.193331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:36.193688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:36.195049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:36.207365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 639/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:36.213388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:36.248752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:36.324710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743b69-46f8-4155-8c3b-932fd2ecee0d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:36.367240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:36.368071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:36.368355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:36.368634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:36.376175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:36.377607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 03:47:36.393465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:36.401750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:36.402923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 641/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:36.403843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:36.404718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:36.501217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ebfd0-1901-4578-a676-e0708555342e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:36.542454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:36.543125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:36.543473 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:36.543832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:36.544683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:36.544986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:36.546416 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:36.566869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 640/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 1 03:47:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:36.572938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:36.667684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:36.677408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32265f-083e-45ba-99ca-0b4cf0c73ac8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:36.711912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0841c-3a82-4b32-be59-08a9ed0ae1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:36.712558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0841c-3a82-4b32-be59-08a9ed0ae1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:36.712840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0841c-3a82-4b32-be59-08a9ed0ae1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:36.713117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0841c-3a82-4b32-be59-08a9ed0ae1d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:36.780557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0841c-3a82-4b32-be59-08a9ed0ae1d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:36.781930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0841c-3a82-4b32-be59-08a9ed0ae1d7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:36.870451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:36.881297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 642/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 1 03:47:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 03:47:36.887376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:36.952650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:36.993719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfea52-9ae9-407f-8391-19b73c078f71 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:37.026237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:37.027010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:37.027316 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:37.027592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:37.049827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:37.049827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 03:47:37.057201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:37.058150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:37.059274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 03:47:37.059274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 03:47:37.059274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 03:47:37.059274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:37.059274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 641/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:37.064793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0841c-3a82-4b32-be59-08a9ed0ae1d7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:37.164637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0841c-3a82-4b32-be59-08a9ed0ae1d7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:37.200661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:37.201356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:37.201731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:37.202012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:37.202733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:37.203050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:37.204456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:37.216055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 643/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 1 03:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:37.221930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:37.327009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bca19-c1b8-4fb1-b0bb-f7e048c754b6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:37.341718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:37.363329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:37.364124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:37.364412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:37.364693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:37.371867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:37.373299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 03:47:37.388283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:37.392330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 01 03:47:37.394408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 642/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 1 03:47:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:37.397499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:37.449727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:37.500887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b74d77-eecc-425b-b669-c0ad383dece0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:37.535077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacbeb4-b705-4657-9510-7751a5257d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:37.535778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacbeb4-b705-4657-9510-7751a5257d8c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:37.536063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacbeb4-b705-4657-9510-7751a5257d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:37.536379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacbeb4-b705-4657-9510-7751a5257d8c None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:37.537173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacbeb4-b705-4657-9510-7751a5257d8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:37.537484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacbeb4-b705-4657-9510-7751a5257d8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:37.538903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacbeb4-b705-4657-9510-7751a5257d8c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:37.552641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 644/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 1 03:47:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:37.559140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:37.660876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb4437-6cdb-41e3-91dc-df5064de967b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:37.696775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d15183-e114-4f15-b16a-791a0ca202e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:37.697430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15183-e114-4f15-b16a-791a0ca202e4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:37.697712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15183-e114-4f15-b16a-791a0ca202e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:37.697988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15183-e114-4f15-b16a-791a0ca202e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:37.698819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15183-e114-4f15-b16a-791a0ca202e4 None admin] RBAC: Adding query filter params (role.id=2e3fcdfa6cc7468b840e70684e783ca1, scope.project.id=6bcddee341bf4069b0154f2eceb30cf3, user.id=f408d50bfc3a4366aabb1cdf86da7935) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:37.699141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15183-e114-4f15-b16a-791a0ca202e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:37.700656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15183-e114-4f15-b16a-791a0ca202e4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:37.707015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 643/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 1 03:47:37 2022] GET /identity/v3/role_assignments?user.id=f408d50bfc3a4366aabb1cdf86da7935&scope.project.id=6bcddee341bf4069b0154f2eceb30cf3&role.id=2e3fcdfa6cc7468b840e70684e783ca1 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:39.802515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de626d0a-05c5-4000-999c-373c03533e48 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:39.803043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de626d0a-05c5-4000-999c-373c03533e48 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:39.803163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de626d0a-05c5-4000-999c-373c03533e48 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:39.805308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 645/1288] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:39.810497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944379ca-99f9-4783-8404-199c594161c4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:39.810934 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944379ca-99f9-4783-8404-199c594161c4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:39.811050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944379ca-99f9-4783-8404-199c594161c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:39.829210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-944379ca-99f9-4783-8404-199c594161c4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:39.830430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-944379ca-99f9-4783-8404-199c594161c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:39.861953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:39.900530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 644/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:39.924229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de626d0a-05c5-4000-999c-373c03533e48 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:40.027477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de626d0a-05c5-4000-999c-373c03533e48 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:40.039145 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:40.063242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:40.064063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:40.064351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:40.064630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:40.065287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:40.066800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:40.080461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:40.081810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:40.082729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 646/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:47:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:40.088048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944379ca-99f9-4783-8404-199c594161c4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:40.191535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944379ca-99f9-4783-8404-199c594161c4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:40.228347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88780d-6565-467e-be1c-c1a59c5c4800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:40.229007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88780d-6565-467e-be1c-c1a59c5c4800 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:40.229291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88780d-6565-467e-be1c-c1a59c5c4800 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:40.229569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88780d-6565-467e-be1c-c1a59c5c4800 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:40.230281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88780d-6565-467e-be1c-c1a59c5c4800 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:40.230618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88780d-6565-467e-be1c-c1a59c5c4800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:40.232288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88780d-6565-467e-be1c-c1a59c5c4800 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:40.238184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 645/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Jul 1 03:47:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:40.244134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:40.360320 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3752a-749d-4a6b-ba52-aa358d6888a1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:40.404704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0084f-77cc-48e3-a5a7-4343ed07d88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:40.405361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0084f-77cc-48e3-a5a7-4343ed07d88b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:40.405633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0084f-77cc-48e3-a5a7-4343ed07d88b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:40.405909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0084f-77cc-48e3-a5a7-4343ed07d88b None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:40.407096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0084f-77cc-48e3-a5a7-4343ed07d88b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:40.407438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0084f-77cc-48e3-a5a7-4343ed07d88b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:40.408822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0084f-77cc-48e3-a5a7-4343ed07d88b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:40.413431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 647/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Jul 1 03:47:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:42.453705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03657c7-ac78-4a79-bac9-f06f3da9d0fa None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:42.454236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03657c7-ac78-4a79-bac9-f06f3da9d0fa None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:42.454236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03657c7-ac78-4a79-bac9-f06f3da9d0fa None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:42.455645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 646/1293] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:42.460786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb2834-4a4c-4ced-aee8-229b967ebdf3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:42.461213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb2834-4a4c-4ced-aee8-229b967ebdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:42.461348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb2834-4a4c-4ced-aee8-229b967ebdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:42.477946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-92bb2834-4a4c-4ced-aee8-229b967ebdf3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:42.479140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-92bb2834-4a4c-4ced-aee8-229b967ebdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:42.510473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:42.553077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 648/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 03:47:42.577073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03657c7-ac78-4a79-bac9-f06f3da9d0fa None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:42.609366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:42.684221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03657c7-ac78-4a79-bac9-f06f3da9d0fa None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:42.720150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be6949-45aa-45cf-93e2-1633246bc076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:42.720810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be6949-45aa-45cf-93e2-1633246bc076 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:42.721089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be6949-45aa-45cf-93e2-1633246bc076 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:42.721357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be6949-45aa-45cf-93e2-1633246bc076 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:42.722123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be6949-45aa-45cf-93e2-1633246bc076 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:42.723504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be6949-45aa-45cf-93e2-1633246bc076 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:42.853793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 647/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 1 03:47:42 2022] POST /identity/v3/services => generated 246 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:47:44.948751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902631b2-e76b-4f47-94c6-ce8ffb05d568 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:44.949282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902631b2-e76b-4f47-94c6-ce8ffb05d568 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:44.949420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902631b2-e76b-4f47-94c6-ce8ffb05d568 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:44.950692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 649/1296] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:44.956692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96db63-bcae-4ea7-a0cd-67c5e918186f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:44.957120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96db63-bcae-4ea7-a0cd-67c5e918186f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:44.957303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96db63-bcae-4ea7-a0cd-67c5e918186f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:44.973805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3f96db63-bcae-4ea7-a0cd-67c5e918186f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:44.975005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3f96db63-bcae-4ea7-a0cd-67c5e918186f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:45.052727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 648/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:45.077063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902631b2-e76b-4f47-94c6-ce8ffb05d568 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:45.183132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902631b2-e76b-4f47-94c6-ce8ffb05d568 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:45.220155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:45.220998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:45.221283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:45.221572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:45.222223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:45.223624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:45.236417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:45.237459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:45.238504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 650/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:47:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:45.243979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96db63-bcae-4ea7-a0cd-67c5e918186f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:45.264382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:45.347722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96db63-bcae-4ea7-a0cd-67c5e918186f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:45.384412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:45.385082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:45.385365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:45.385640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:45.386355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:45.386660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:45.388166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:45.393964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 649/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Jul 1 03:47:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:45.400138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:45.433333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:45.502878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482abb2-53ab-43b6-ae3b-937a8d21a1a6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:45.537865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b51e29-de16-44c6-84d1-d2583132a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:45.538523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b51e29-de16-44c6-84d1-d2583132a3ad None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:45.538806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b51e29-de16-44c6-84d1-d2583132a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:45.539085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b51e29-de16-44c6-84d1-d2583132a3ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:45.539900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b51e29-de16-44c6-84d1-d2583132a3ad None admin] RBAC: Adding query filter params (interface=public, service_id=25eb5990018646c4a293d1be5c259a45, region_id=RegionOne) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:45.540218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b51e29-de16-44c6-84d1-d2583132a3ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:45.541578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b51e29-de16-44c6-84d1-d2583132a3ad None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:45.546730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 651/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 1 03:47:45 2022] GET /identity/v3/endpoints?service_id=25eb5990018646c4a293d1be5c259a45&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:45.552523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:45.647019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:45.666180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53080033-2b04-43d6-8c77-a09f5035caeb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:45.704025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a43f6-209b-4ecd-8efc-3d9282831ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:45.704704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a43f6-209b-4ecd-8efc-3d9282831ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:45.704985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a43f6-209b-4ecd-8efc-3d9282831ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:45.705263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a43f6-209b-4ecd-8efc-3d9282831ca8 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:45.705956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a43f6-209b-4ecd-8efc-3d9282831ca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:45.707360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a43f6-209b-4ecd-8efc-3d9282831ca8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:45.714928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 650/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 1 03:47:45 2022] GET /identity/v3/services => generated 2191 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 03:47:47.763583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa140a-2ce8-42b0-9ff3-5e350253a9fd None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:47.764112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa140a-2ce8-42b0-9ff3-5e350253a9fd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:47.764112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa140a-2ce8-42b0-9ff3-5e350253a9fd None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:47.766018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 652/1302] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:47:47.771481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefea9e-bfed-4cb7-a3c1-f3c16fb952db None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:47.771960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefea9e-bfed-4cb7-a3c1-f3c16fb952db None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:47.772063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefea9e-bfed-4cb7-a3c1-f3c16fb952db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:47.789883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7aefea9e-bfed-4cb7-a3c1-f3c16fb952db None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:47:47.792549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7aefea9e-bfed-4cb7-a3c1-f3c16fb952db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:47:47.860563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 651/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:47:47 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 03:47:47.883994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa140a-2ce8-42b0-9ff3-5e350253a9fd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:47.983058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:47.989911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa140a-2ce8-42b0-9ff3-5e350253a9fd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:48.025026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:48.025820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:48.026104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:48.026381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:48.027037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:48.028681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 03:47:48.041578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 03:47:48.043866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 01 03:47:48.044930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 653/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 1 03:47:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 03:47:48.049960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefea9e-bfed-4cb7-a3c1-f3c16fb952db None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:48.130835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:48.154693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefea9e-bfed-4cb7-a3c1-f3c16fb952db None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:48.189441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60351acb-27ef-4763-a498-9de4cb182083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:48.190300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60351acb-27ef-4763-a498-9de4cb182083 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:48.190618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60351acb-27ef-4763-a498-9de4cb182083 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:48.191297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60351acb-27ef-4763-a498-9de4cb182083 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:48.192014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60351acb-27ef-4763-a498-9de4cb182083 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 03:47:48.192329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60351acb-27ef-4763-a498-9de4cb182083 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:48.193732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60351acb-27ef-4763-a498-9de4cb182083 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:48.199484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 652/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Jul 1 03:47:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 03:47:48.205418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:47:48.309218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cbbee-fea3-4183-8533-ed79975d37eb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:47:48.343938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef437aa-d930-47b6-a6fd-9cd20a0cd9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:47:48.344637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef437aa-d930-47b6-a6fd-9cd20a0cd9f0 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:47:48.344922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef437aa-d930-47b6-a6fd-9cd20a0cd9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:47:48.345202 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef437aa-d930-47b6-a6fd-9cd20a0cd9f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:47:48.345989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef437aa-d930-47b6-a6fd-9cd20a0cd9f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:47:48.347382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef437aa-d930-47b6-a6fd-9cd20a0cd9f0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:47:48.505429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 654/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 1 03:47:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 03:47:58.195986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=30 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:47:58.498283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:34.628183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13933b-9a15-479d-ac5f-d6e53ff2365e None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:34.629115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13933b-9a15-479d-ac5f-d6e53ff2365e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:34.629545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13933b-9a15-479d-ac5f-d6e53ff2365e None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:34.636876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70d6c-f899-425a-b475-621408e4e9d8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:34.637780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70d6c-f899-425a-b475-621408e4e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:34.638212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70d6c-f899-425a-b475-621408e4e9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:34.642376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 653/1307] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:55:34 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:34.762392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-42a70d6c-f899-425a-b475-621408e4e9d8 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:34.764836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-42a70d6c-f899-425a-b475-621408e4e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:35.047757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 655/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 03:55:35.103118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efeb7-c535-43c1-bb77-9061756708f6 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:35.103489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efeb7-c535-43c1-bb77-9061756708f6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:35.103778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efeb7-c535-43c1-bb77-9061756708f6 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:35.105232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 654/1309] 10.222.0.20 () {54 vars in 964 bytes} [Fri Jul 1 03:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:35.113622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d4c07-f691-4319-bd71-4cd4e769a4e0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:35.114072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d4c07-f691-4319-bd71-4cd4e769a4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:35.114484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d4c07-f691-4319-bd71-4cd4e769a4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:35.264508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8c2d4c07-f691-4319-bd71-4cd4e769a4e0 None None] MFA Rules not processed for user `0beff646206e446386d91537df9eaa67`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:35.266326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8c2d4c07-f691-4319-bd71-4cd4e769a4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:35.294514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:35.359308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 656/1310] 10.222.0.20 () {58 vars in 1061 bytes} [Fri Jul 1 03:55:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 03:55:35.369565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9efeb7-c535-43c1-bb77-9061756708f6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:55:35.372953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-dd9efeb7-c535-43c1-bb77-9061756708f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:35.500576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9efeb7-c535-43c1-bb77-9061756708f6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:55:35.590427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720beb4-f3f8-4280-be8b-1a4e0bee98cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:55:35.591474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720beb4-f3f8-4280-be8b-1a4e0bee98cc service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:35.591968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720beb4-f3f8-4280-be8b-1a4e0bee98cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:35.592430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720beb4-f3f8-4280-be8b-1a4e0bee98cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:35.594299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7720beb4-f3f8-4280-be8b-1a4e0bee98cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:35.660185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:35.749414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720beb4-f3f8-4280-be8b-1a4e0bee98cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:55:35.751478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720beb4-f3f8-4280-be8b-1a4e0bee98cc service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:55:35.781896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 655/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:55:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 03:55:39.398789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143d59-d42d-4a8d-a0d0-87e6af08512f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:39.399466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143d59-d42d-4a8d-a0d0-87e6af08512f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:39.399767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143d59-d42d-4a8d-a0d0-87e6af08512f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:39.402443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 657/1312] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:55:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:39.408347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c077f87-30aa-4c45-8e97-7ba13666f82b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:39.408921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c077f87-30aa-4c45-8e97-7ba13666f82b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:39.408921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c077f87-30aa-4c45-8e97-7ba13666f82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:39.490873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7c077f87-30aa-4c45-8e97-7ba13666f82b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:39.492968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7c077f87-30aa-4c45-8e97-7ba13666f82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:39.670912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 656/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:55:39.713570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2143d59-d42d-4a8d-a0d0-87e6af08512f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:55:39.715394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a2143d59-d42d-4a8d-a0d0-87e6af08512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:39.861243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2143d59-d42d-4a8d-a0d0-87e6af08512f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:55:39.954141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:39.957976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bab8fe-31d7-4033-bac4-d84ed2af450e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:55:39.958713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bab8fe-31d7-4033-bac4-d84ed2af450e service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:39.959046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bab8fe-31d7-4033-bac4-d84ed2af450e service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:39.959408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bab8fe-31d7-4033-bac4-d84ed2af450e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:39.961482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-52bab8fe-31d7-4033-bac4-d84ed2af450e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:40.089297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bab8fe-31d7-4033-bac4-d84ed2af450e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:55:40.091030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bab8fe-31d7-4033-bac4-d84ed2af450e service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:55:40.113077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 658/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 03:55:42.663544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc67886-79d7-4d15-9c87-5b4f1379ddb5 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:42.664121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc67886-79d7-4d15-9c87-5b4f1379ddb5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:42.664121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc67886-79d7-4d15-9c87-5b4f1379ddb5 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:42.665463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 657/1315] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:42.670626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b784501-dc47-4e72-9cfa-2edce99ca8a4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:42.671037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b784501-dc47-4e72-9cfa-2edce99ca8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:42.671181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b784501-dc47-4e72-9cfa-2edce99ca8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:42.734841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2b784501-dc47-4e72-9cfa-2edce99ca8a4 None None] MFA Rules not processed for user `9143316c5cac437da00eea03db182f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:42.736106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2b784501-dc47-4e72-9cfa-2edce99ca8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:42.786881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:42.829505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 659/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:55:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:55:42.870967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc67886-79d7-4d15-9c87-5b4f1379ddb5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:55:42.872553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7fc67886-79d7-4d15-9c87-5b4f1379ddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:42.882783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:42.975316 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc67886-79d7-4d15-9c87-5b4f1379ddb5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:55:43.048525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bb3f2-3254-4cd4-aabc-ca3e3b3f161c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:55:43.049599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bb3f2-3254-4cd4-aabc-ca3e3b3f161c service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:43.049878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bb3f2-3254-4cd4-aabc-ca3e3b3f161c service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:43.050151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bb3f2-3254-4cd4-aabc-ca3e3b3f161c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:43.051849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-069bb3f2-3254-4cd4-aabc-ca3e3b3f161c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:43.115614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:43.166146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bb3f2-3254-4cd4-aabc-ca3e3b3f161c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:55:43.167661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bb3f2-3254-4cd4-aabc-ca3e3b3f161c service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:55:43.187077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 658/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:55:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:55:46.156062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953174-d0e1-4275-802e-e78ddee68b23 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:46.156596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953174-d0e1-4275-802e-e78ddee68b23 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:46.156689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953174-d0e1-4275-802e-e78ddee68b23 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:46.158118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 660/1318] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:46.163506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6834d-db1b-4c20-a7cb-5f9a650f2910 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:46.163969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6834d-db1b-4c20-a7cb-5f9a650f2910 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:46.164099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6834d-db1b-4c20-a7cb-5f9a650f2910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:46.230400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6db6834d-db1b-4c20-a7cb-5f9a650f2910 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:46.231669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6db6834d-db1b-4c20-a7cb-5f9a650f2910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:46.350732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 659/1319] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:55:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:55:46.391311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953174-d0e1-4275-802e-e78ddee68b23 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:55:46.392865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b5953174-d0e1-4275-802e-e78ddee68b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:46.505680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953174-d0e1-4275-802e-e78ddee68b23 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:55:46.584689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb6f30-0b4e-4878-85f0-4f21314e3f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:55:46.585369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb6f30-0b4e-4878-85f0-4f21314e3f8e service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:46.585647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb6f30-0b4e-4878-85f0-4f21314e3f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:46.585934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb6f30-0b4e-4878-85f0-4f21314e3f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:46.587632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-11bb6f30-0b4e-4878-85f0-4f21314e3f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:46.594869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:46.714595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb6f30-0b4e-4878-85f0-4f21314e3f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:55:46.716140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb6f30-0b4e-4878-85f0-4f21314e3f8e service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:55:46.737815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 661/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:55:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:55:49.945189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b17e1-a9cb-4c47-a0f4-d65aa6f1f255 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:49.945768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b17e1-a9cb-4c47-a0f4-d65aa6f1f255 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:49.945768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b17e1-a9cb-4c47-a0f4-d65aa6f1f255 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:49.947118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 660/1321] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:55:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:49.952958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b02eb-d810-45c0-b83e-e4d15e0b7533 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:49.953537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b02eb-d810-45c0-b83e-e4d15e0b7533 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:49.953651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b02eb-d810-45c0-b83e-e4d15e0b7533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:50.019888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-458b02eb-d810-45c0-b83e-e4d15e0b7533 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:50.021126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-458b02eb-d810-45c0-b83e-e4d15e0b7533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:50.081214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:50.121657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 662/1322] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:55:50.157512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b17e1-a9cb-4c47-a0f4-d65aa6f1f255 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:55:50.159697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b81b17e1-a9cb-4c47-a0f4-d65aa6f1f255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:50.182053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:50.300952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b17e1-a9cb-4c47-a0f4-d65aa6f1f255 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:55:50.391704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cef08-a329-43ce-b417-b7037dcb694f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:55:50.392930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cef08-a329-43ce-b417-b7037dcb694f service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:50.393217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cef08-a329-43ce-b417-b7037dcb694f service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:50.393494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cef08-a329-43ce-b417-b7037dcb694f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:50.395355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-207cef08-a329-43ce-b417-b7037dcb694f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:50.477061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:50.520958 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cef08-a329-43ce-b417-b7037dcb694f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:55:50.522414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cef08-a329-43ce-b417-b7037dcb694f service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:55:50.545538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 661/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:55:52.695001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312cc0c-6318-4ba0-beea-de127532d14e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:52.695578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312cc0c-6318-4ba0-beea-de127532d14e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:52.695578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312cc0c-6318-4ba0-beea-de127532d14e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:52.697126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 663/1324] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:52.702749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648236c0-9e81-41c9-9b3c-5ed0c87e06e2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:52.703332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648236c0-9e81-41c9-9b3c-5ed0c87e06e2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:52.703332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648236c0-9e81-41c9-9b3c-5ed0c87e06e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:52.786623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-648236c0-9e81-41c9-9b3c-5ed0c87e06e2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:52.787913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-648236c0-9e81-41c9-9b3c-5ed0c87e06e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:52.913664 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 662/1325] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:55:52.950412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312cc0c-6318-4ba0-beea-de127532d14e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:55:52.952042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9312cc0c-6318-4ba0-beea-de127532d14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:53.068405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312cc0c-6318-4ba0-beea-de127532d14e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:55:53.161391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6c891-6ad8-44bf-8f55-152b498021d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:55:53.162073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c891-6ad8-44bf-8f55-152b498021d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:53.162358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c891-6ad8-44bf-8f55-152b498021d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:53.162639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c891-6ad8-44bf-8f55-152b498021d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:53.164492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2fa6c891-6ad8-44bf-8f55-152b498021d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:53.184522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:53.294054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6c891-6ad8-44bf-8f55-152b498021d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:55:53.295548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6c891-6ad8-44bf-8f55-152b498021d9 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:55:53.324329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 664/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:55:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:55:55.450104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593988-1f9f-4406-a65c-c5c4b6a5fd8f None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:55.450643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593988-1f9f-4406-a65c-c5c4b6a5fd8f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:55.450643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593988-1f9f-4406-a65c-c5c4b6a5fd8f None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:55.452060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 663/1327] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:55.457769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2273-3ff3-43a0-a739-8bdcdaa2de07 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:55.458281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2273-3ff3-43a0-a739-8bdcdaa2de07 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:55.458491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2273-3ff3-43a0-a739-8bdcdaa2de07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:55.531240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d4fd2273-3ff3-43a0-a739-8bdcdaa2de07 None None] MFA Rules not processed for user `9143316c5cac437da00eea03db182f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:55.532499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d4fd2273-3ff3-43a0-a739-8bdcdaa2de07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:55.609469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:55.647265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 665/1328] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:55:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:55:55.683506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f593988-1f9f-4406-a65c-c5c4b6a5fd8f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:55:55.685410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2f593988-1f9f-4406-a65c-c5c4b6a5fd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:55.727660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:55.816095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f593988-1f9f-4406-a65c-c5c4b6a5fd8f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:55:55.904194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713a6a-da60-41c6-a5a5-2bfd8cab4e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:55:55.904866 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713a6a-da60-41c6-a5a5-2bfd8cab4e66 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:55.905139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713a6a-da60-41c6-a5a5-2bfd8cab4e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:55.905413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713a6a-da60-41c6-a5a5-2bfd8cab4e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:55.907202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-94713a6a-da60-41c6-a5a5-2bfd8cab4e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:56.013035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:56.053850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94713a6a-da60-41c6-a5a5-2bfd8cab4e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:55:56.056702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94713a6a-da60-41c6-a5a5-2bfd8cab4e66 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:55:56.080631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 664/1329] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:55:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:55:58.668187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f40f0a-4f3d-4537-b8c4-c9375dc6d0fb None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:58.668939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f40f0a-4f3d-4537-b8c4-c9375dc6d0fb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:58.668939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f40f0a-4f3d-4537-b8c4-c9375dc6d0fb None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:58.670199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 666/1330] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:55:58.676101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142028d8-9c45-49e0-bff4-13e2d15b7b27 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:58.676623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142028d8-9c45-49e0-bff4-13e2d15b7b27 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:58.676735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142028d8-9c45-49e0-bff4-13e2d15b7b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:58.751218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-142028d8-9c45-49e0-bff4-13e2d15b7b27 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:55:58.752503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-142028d8-9c45-49e0-bff4-13e2d15b7b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:58.856289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 665/1331] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:55:58.893291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f40f0a-4f3d-4537-b8c4-c9375dc6d0fb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:55:58.894848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-13f40f0a-4f3d-4537-b8c4-c9375dc6d0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:58.999491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f40f0a-4f3d-4537-b8c4-c9375dc6d0fb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:55:59.075001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a23a7d-f16c-4219-8814-743e63699fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:55:59.075687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a23a7d-f16c-4219-8814-743e63699fed service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:55:59.075966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a23a7d-f16c-4219-8814-743e63699fed service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:55:59.076236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a23a7d-f16c-4219-8814-743e63699fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:55:59.077911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-70a23a7d-f16c-4219-8814-743e63699fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:55:59.108730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:55:59.208933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a23a7d-f16c-4219-8814-743e63699fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:55:59.210383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a23a7d-f16c-4219-8814-743e63699fed service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:55:59.232098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 667/1332] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:01.324810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd332b-e000-4503-9ebd-df2c476e7381 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:01.325413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd332b-e000-4503-9ebd-df2c476e7381 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:01.325413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd332b-e000-4503-9ebd-df2c476e7381 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:01.326717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 666/1333] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:01.332592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fac397-07e3-4ace-b6c5-0010c9c8b0d5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:01.333036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fac397-07e3-4ace-b6c5-0010c9c8b0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:01.333334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fac397-07e3-4ace-b6c5-0010c9c8b0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:01.399645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-27fac397-07e3-4ace-b6c5-0010c9c8b0d5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:01.400897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-27fac397-07e3-4ace-b6c5-0010c9c8b0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:01.479301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:01.502368 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 668/1334] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:01.543077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd332b-e000-4503-9ebd-df2c476e7381 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:01.544704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e5dd332b-e000-4503-9ebd-df2c476e7381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:01.586141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:01.658023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd332b-e000-4503-9ebd-df2c476e7381 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:01.734818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496b336-674d-4529-b082-533f8388d0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:01.735501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496b336-674d-4529-b082-533f8388d0fc service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:01.736292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496b336-674d-4529-b082-533f8388d0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:01.736562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496b336-674d-4529-b082-533f8388d0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:01.738247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3496b336-674d-4529-b082-533f8388d0fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:01.824600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:01.855489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496b336-674d-4529-b082-533f8388d0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:01.856902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496b336-674d-4529-b082-533f8388d0fc service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:01.877910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 667/1335] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:04.542842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b4537-f20d-4082-8cc4-3beeac439de5 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:04.543276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b4537-f20d-4082-8cc4-3beeac439de5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:04.543518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b4537-f20d-4082-8cc4-3beeac439de5 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:04.544915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 669/1336] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:04.551104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45af0f-304a-4ce1-923c-4c154a7e8ffe None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:04.551651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45af0f-304a-4ce1-923c-4c154a7e8ffe None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:04.551756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45af0f-304a-4ce1-923c-4c154a7e8ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:04.615847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6a45af0f-304a-4ce1-923c-4c154a7e8ffe None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:04.617330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6a45af0f-304a-4ce1-923c-4c154a7e8ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:04.719251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 668/1337] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:04.770662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b30f5-819f-4288-9a64-200ccf7058f2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:04.770954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b30f5-819f-4288-9a64-200ccf7058f2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:04.771279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b30f5-819f-4288-9a64-200ccf7058f2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:04.772661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 670/1338] 10.222.0.20 () {54 vars in 964 bytes} [Fri Jul 1 03:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:04.778657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a6179-110d-4dbd-9827-f74901751cce None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:04.778946 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a6179-110d-4dbd-9827-f74901751cce None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:04.779624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a6179-110d-4dbd-9827-f74901751cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:04.833670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:04.857703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3c6a6179-110d-4dbd-9827-f74901751cce None None] MFA Rules not processed for user `0beff646206e446386d91537df9eaa67`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:04.859056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3c6a6179-110d-4dbd-9827-f74901751cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:04.940217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 669/1339] 10.222.0.20 () {58 vars in 1061 bytes} [Fri Jul 1 03:56:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:04.949941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b30f5-819f-4288-9a64-200ccf7058f2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:04.953161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e27b30f5-819f-4288-9a64-200ccf7058f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:05.078713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b30f5-819f-4288-9a64-200ccf7058f2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:05.082561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb90bcd-5b3a-4676-a0fa-c3c73a64b82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:05.083248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb90bcd-5b3a-4676-a0fa-c3c73a64b82c service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:05.083547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb90bcd-5b3a-4676-a0fa-c3c73a64b82c service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:05.083880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb90bcd-5b3a-4676-a0fa-c3c73a64b82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:05.085751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8bb90bcd-5b3a-4676-a0fa-c3c73a64b82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:05.197744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb90bcd-5b3a-4676-a0fa-c3c73a64b82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:05.199246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb90bcd-5b3a-4676-a0fa-c3c73a64b82c service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:05.210774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:05.221386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 671/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:10.138977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fccbb-b051-4c64-bd4a-07b31d3fe5ac None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:10.145748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fccbb-b051-4c64-bd4a-07b31d3fe5ac None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:10.145748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fccbb-b051-4c64-bd4a-07b31d3fe5ac None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:10.145748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 670/1341] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:10.147791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717acba-21e2-46e9-9191-cbc097209b59 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:10.148251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717acba-21e2-46e9-9191-cbc097209b59 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:10.148478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717acba-21e2-46e9-9191-cbc097209b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:10.212598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9717acba-21e2-46e9-9191-cbc097209b59 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:10.214308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9717acba-21e2-46e9-9191-cbc097209b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:10.315742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 672/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:10.357508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fccbb-b051-4c64-bd4a-07b31d3fe5ac None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:10.370059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fccbb-b051-4c64-bd4a-07b31d3fe5ac None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:10.374178 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72101fa7-1666-427a-8252-c5d57de7bfbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:10.377296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72101fa7-1666-427a-8252-c5d57de7bfbb service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:10.377296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72101fa7-1666-427a-8252-c5d57de7bfbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:10.377296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72101fa7-1666-427a-8252-c5d57de7bfbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:10.386031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72101fa7-1666-427a-8252-c5d57de7bfbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:10.387665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72101fa7-1666-427a-8252-c5d57de7bfbb service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:10.399309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 671/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:13.593497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c309f7-c644-47d6-8c2f-9ea665d0c092 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:13.594058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c309f7-c644-47d6-8c2f-9ea665d0c092 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:13.594058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c309f7-c644-47d6-8c2f-9ea665d0c092 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:13.595559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 673/1344] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:13.602177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af567d8-5325-4266-9d7b-da5bdd96fa71 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:13.603453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af567d8-5325-4266-9d7b-da5bdd96fa71 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:13.603453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af567d8-5325-4266-9d7b-da5bdd96fa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:13.635093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4af567d8-5325-4266-9d7b-da5bdd96fa71 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:13.636519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4af567d8-5325-4266-9d7b-da5bdd96fa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:13.653272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 672/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:13.692758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c309f7-c644-47d6-8c2f-9ea665d0c092 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:13.706730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c309f7-c644-47d6-8c2f-9ea665d0c092 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:13.710785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36c417-ae68-4af3-951d-93f41783881c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:13.711567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c417-ae68-4af3-951d-93f41783881c service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:13.711911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c417-ae68-4af3-951d-93f41783881c service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:13.712253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c417-ae68-4af3-951d-93f41783881c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:13.723661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36c417-ae68-4af3-951d-93f41783881c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:13.725427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36c417-ae68-4af3-951d-93f41783881c service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:13.736653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 674/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 03:56:18.163972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b88d-255f-4fdf-9bf0-05eae9aea309 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:18.164740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b88d-255f-4fdf-9bf0-05eae9aea309 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:18.164740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b88d-255f-4fdf-9bf0-05eae9aea309 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:18.165908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 673/1347] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:56:18.171674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065762-ea12-4d9a-852a-45b6c25396af None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:18.172229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065762-ea12-4d9a-852a-45b6c25396af None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:18.172229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065762-ea12-4d9a-852a-45b6c25396af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:18.191074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1c065762-ea12-4d9a-852a-45b6c25396af None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:18.192819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1c065762-ea12-4d9a-852a-45b6c25396af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:18.201549 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 675/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:18.243899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564b88d-255f-4fdf-9bf0-05eae9aea309 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:18.257144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564b88d-255f-4fdf-9bf0-05eae9aea309 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:18.261990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729490b-fa79-4ab9-b363-e3b45b31ece0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:18.262791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729490b-fa79-4ab9-b363-e3b45b31ece0 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:18.263078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729490b-fa79-4ab9-b363-e3b45b31ece0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:18.263566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729490b-fa79-4ab9-b363-e3b45b31ece0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:18.275020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729490b-fa79-4ab9-b363-e3b45b31ece0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:18.276932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729490b-fa79-4ab9-b363-e3b45b31ece0 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:18.288551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 674/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:23.647828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f498-94f5-4684-a599-16ae4ce9a4d4 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:23.652839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f498-94f5-4684-a599-16ae4ce9a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:23.653583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f498-94f5-4684-a599-16ae4ce9a4d4 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:23.655174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 676/1350] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 03:56:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:23.665320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368fcfa-9eb9-47ea-af60-de82e01a8886 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:23.665886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368fcfa-9eb9-47ea-af60-de82e01a8886 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:23.666357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368fcfa-9eb9-47ea-af60-de82e01a8886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:23.779182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a368fcfa-9eb9-47ea-af60-de82e01a8886 None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:23.780975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a368fcfa-9eb9-47ea-af60-de82e01a8886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:23.838609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:23.882461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 675/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 03:56:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:28.185861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=56 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:29.592628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a86f6-931a-40cc-99d1-546311e35d0b None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:29.593669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a86f6-931a-40cc-99d1-546311e35d0b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:29.594134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a86f6-931a-40cc-99d1-546311e35d0b None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:29.596067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 677/1352] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 03:56:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:29.603033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43717a-a082-4df9-844c-32fe8e349dea None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:29.603585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43717a-a082-4df9-844c-32fe8e349dea None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:29.604094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43717a-a082-4df9-844c-32fe8e349dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:29.642674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4c43717a-a082-4df9-844c-32fe8e349dea None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:29.644973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4c43717a-a082-4df9-844c-32fe8e349dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:29.665077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 676/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 03:56:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:31.266804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784f8b-9112-4504-b0b2-05d28733d09e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:31.267779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784f8b-9112-4504-b0b2-05d28733d09e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:31.267779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784f8b-9112-4504-b0b2-05d28733d09e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:31.269010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 678/1354] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 03:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:31.274754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f2329-d070-4664-ac74-cb39123dd31c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:31.275344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f2329-d070-4664-ac74-cb39123dd31c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:31.275507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f2329-d070-4664-ac74-cb39123dd31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:31.296946 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-285f2329-d070-4664-ac74-cb39123dd31c None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:31.298711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-285f2329-d070-4664-ac74-cb39123dd31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:31.307745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 677/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 03:56:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:35.299866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454584d-53df-459d-9af6-5af67fbe0cea None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:35.300357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454584d-53df-459d-9af6-5af67fbe0cea None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:35.300463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454584d-53df-459d-9af6-5af67fbe0cea None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:35.301785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 679/1356] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:35.308662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005aee-8f05-4d84-934e-e9c9dfe3bd9e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:35.309428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005aee-8f05-4d84-934e-e9c9dfe3bd9e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:35.309428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005aee-8f05-4d84-934e-e9c9dfe3bd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:35.331373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-dc005aee-8f05-4d84-934e-e9c9dfe3bd9e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:35.332584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-dc005aee-8f05-4d84-934e-e9c9dfe3bd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:35.340724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 678/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:35.407883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae134-1c33-4018-a7ea-19cc5cfa7fdb None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:35.408596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae134-1c33-4018-a7ea-19cc5cfa7fdb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:35.409131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae134-1c33-4018-a7ea-19cc5cfa7fdb None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:35.410920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 680/1358] 10.222.0.20 () {54 vars in 961 bytes} [Fri Jul 1 03:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:35.423040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508353ad-4f88-42a7-9741-8f1f19502c0f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:35.425000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508353ad-4f88-42a7-9741-8f1f19502c0f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:35.425660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508353ad-4f88-42a7-9741-8f1f19502c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:35.509633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-508353ad-4f88-42a7-9741-8f1f19502c0f None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:35.512583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-508353ad-4f88-42a7-9741-8f1f19502c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:35.644994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 679/1359] 10.222.0.20 () {58 vars in 1058 bytes} [Fri Jul 1 03:56:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:35.653420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dae134-1c33-4018-a7ea-19cc5cfa7fdb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:35.669441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dae134-1c33-4018-a7ea-19cc5cfa7fdb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:35.675389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2006ca-3496-4032-b145-76c202ecd06f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:35.676140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2006ca-3496-4032-b145-76c202ecd06f service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:35.676424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2006ca-3496-4032-b145-76c202ecd06f service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:35.676698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2006ca-3496-4032-b145-76c202ecd06f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:35.686783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2006ca-3496-4032-b145-76c202ecd06f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:35.688434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2006ca-3496-4032-b145-76c202ecd06f service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:35.712044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 681/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:38.047861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24a9a-5e8b-45a8-8859-4b34436e9b91 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:38.048724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24a9a-5e8b-45a8-8859-4b34436e9b91 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:38.048724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24a9a-5e8b-45a8-8859-4b34436e9b91 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:38.050507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 680/1361] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:38.056946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095803-2a5b-40ce-92ca-e4f1fd80b96f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:38.057498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095803-2a5b-40ce-92ca-e4f1fd80b96f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:38.058260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095803-2a5b-40ce-92ca-e4f1fd80b96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:38.088843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fd095803-2a5b-40ce-92ca-e4f1fd80b96f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:38.091011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fd095803-2a5b-40ce-92ca-e4f1fd80b96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:38.104482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 682/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:38.172154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb24a9a-5e8b-45a8-8859-4b34436e9b91 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:38.194990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb24a9a-5e8b-45a8-8859-4b34436e9b91 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:38.205640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cfa3e-8e06-48f7-b48c-04a4e01f973a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:38.206793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cfa3e-8e06-48f7-b48c-04a4e01f973a service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:38.207351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cfa3e-8e06-48f7-b48c-04a4e01f973a service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:38.207851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cfa3e-8e06-48f7-b48c-04a4e01f973a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:38.226269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cfa3e-8e06-48f7-b48c-04a4e01f973a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:38.229447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cfa3e-8e06-48f7-b48c-04a4e01f973a service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:38.250515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 681/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:40.982502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de060f-97b7-4104-9074-acb5292c40fc None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:40.983149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de060f-97b7-4104-9074-acb5292c40fc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:40.983149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de060f-97b7-4104-9074-acb5292c40fc None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:40.984622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 683/1364] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:56:40.990251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e92a4f-1772-4924-a73d-52160c34fb36 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:40.990819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e92a4f-1772-4924-a73d-52160c34fb36 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:40.990819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e92a4f-1772-4924-a73d-52160c34fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:41.013229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-15e92a4f-1772-4924-a73d-52160c34fb36 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:41.014482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-15e92a4f-1772-4924-a73d-52160c34fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:41.023980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 682/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 03:56:41.075056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d403a-033f-40ae-a08d-0d8d3a85db9e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:41.075282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d403a-033f-40ae-a08d-0d8d3a85db9e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:41.075565 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d403a-033f-40ae-a08d-0d8d3a85db9e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:41.076988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 684/1366] 10.222.0.20 () {54 vars in 961 bytes} [Fri Jul 1 03:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:41.082866 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421294c-bca7-4750-9688-3c91ff908ee4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:41.084702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421294c-bca7-4750-9688-3c91ff908ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:41.084702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421294c-bca7-4750-9688-3c91ff908ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:41.104167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a421294c-bca7-4750-9688-3c91ff908ee4 None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:41.105302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a421294c-bca7-4750-9688-3c91ff908ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:41.121407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 683/1367] 10.222.0.20 () {58 vars in 1058 bytes} [Fri Jul 1 03:56:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:41.131744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d403a-033f-40ae-a08d-0d8d3a85db9e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:41.178480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d403a-033f-40ae-a08d-0d8d3a85db9e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:41.187243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f3cb5-116b-4bac-a0b0-7f6f9c82e187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:41.188399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f3cb5-116b-4bac-a0b0-7f6f9c82e187 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:41.188979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f3cb5-116b-4bac-a0b0-7f6f9c82e187 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:41.189396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f3cb5-116b-4bac-a0b0-7f6f9c82e187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:41.201044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f3cb5-116b-4bac-a0b0-7f6f9c82e187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:41.202733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f3cb5-116b-4bac-a0b0-7f6f9c82e187 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:41.214596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 685/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:43.737542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d992901-2a52-4aa6-9dbc-54d2326db9c3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:43.738061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d992901-2a52-4aa6-9dbc-54d2326db9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:43.738061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d992901-2a52-4aa6-9dbc-54d2326db9c3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:43.739463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 684/1369] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:43.745086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed83be2-5324-4f65-a1e8-1e5276b3b86c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:43.745527 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed83be2-5324-4f65-a1e8-1e5276b3b86c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:43.745825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed83be2-5324-4f65-a1e8-1e5276b3b86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:43.766334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4ed83be2-5324-4f65-a1e8-1e5276b3b86c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:43.767907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4ed83be2-5324-4f65-a1e8-1e5276b3b86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:43.776980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 686/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:43.823737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d992901-2a52-4aa6-9dbc-54d2326db9c3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:43.841533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d992901-2a52-4aa6-9dbc-54d2326db9c3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:43.848304 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637b9e1-2f34-4498-a132-457a892ec10b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:43.849283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637b9e1-2f34-4498-a132-457a892ec10b service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:43.849668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637b9e1-2f34-4498-a132-457a892ec10b service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:43.850072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637b9e1-2f34-4498-a132-457a892ec10b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:43.860352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637b9e1-2f34-4498-a132-457a892ec10b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:43.862227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637b9e1-2f34-4498-a132-457a892ec10b service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:43.874477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 685/1371] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:44.545750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2141a-b544-4110-93e1-7e3673c148f0 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:44.548482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2141a-b544-4110-93e1-7e3673c148f0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:44.548771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2141a-b544-4110-93e1-7e3673c148f0 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:44.550903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 687/1372] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:44.556959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499b89-175e-4a3c-8795-b003328fd01c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:44.557603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499b89-175e-4a3c-8795-b003328fd01c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:44.558106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499b89-175e-4a3c-8795-b003328fd01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:44.586570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-13499b89-175e-4a3c-8795-b003328fd01c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:44.588255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-13499b89-175e-4a3c-8795-b003328fd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:44.597379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 686/1373] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:44.912886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cbcd2-5fbc-47ec-aee8-c92a10f35229 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:44.913432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cbcd2-5fbc-47ec-aee8-c92a10f35229 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:44.914124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cbcd2-5fbc-47ec-aee8-c92a10f35229 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:44.916864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 688/1374] 10.222.0.20 () {54 vars in 963 bytes} [Fri Jul 1 03:56:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:44.923707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464d31-af53-41c8-b33c-05e5f6d0c9b6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:44.924415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464d31-af53-41c8-b33c-05e5f6d0c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:44.924415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464d31-af53-41c8-b33c-05e5f6d0c9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:44.971346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:44.989828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-66464d31-af53-41c8-b33c-05e5f6d0c9b6 None None] MFA Rules not processed for user `ec8a755e84f04c9590c9c8d57781d2a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:44.992575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-66464d31-af53-41c8-b33c-05e5f6d0c9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:45.083695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 687/1375] 10.222.0.20 () {58 vars in 1060 bytes} [Fri Jul 1 03:56:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:45.094769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cbcd2-5fbc-47ec-aee8-c92a10f35229 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:45.116792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cbcd2-5fbc-47ec-aee8-c92a10f35229 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:45.122757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2077528-930e-467b-ae29-44b29246a14c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:45.123929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077528-930e-467b-ae29-44b29246a14c service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:45.124412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077528-930e-467b-ae29-44b29246a14c service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:45.124899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077528-930e-467b-ae29-44b29246a14c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:45.136728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2077528-930e-467b-ae29-44b29246a14c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:45.138485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2077528-930e-467b-ae29-44b29246a14c service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:45.153295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 689/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:46.433562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615d5c-b6c2-4ba4-b859-5be60ce7d1dd None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:46.434130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615d5c-b6c2-4ba4-b859-5be60ce7d1dd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:46.434130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615d5c-b6c2-4ba4-b859-5be60ce7d1dd None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:46.435789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 688/1377] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:56:46.441886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9b344-f80e-4385-8dc6-1592f89a30b3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:46.442412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9b344-f80e-4385-8dc6-1592f89a30b3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:46.442412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9b344-f80e-4385-8dc6-1592f89a30b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:46.460979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a8d9b344-f80e-4385-8dc6-1592f89a30b3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:46.462185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a8d9b344-f80e-4385-8dc6-1592f89a30b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:46.470477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 690/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:46.517299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc615d5c-b6c2-4ba4-b859-5be60ce7d1dd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:46.532860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc615d5c-b6c2-4ba4-b859-5be60ce7d1dd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:46.538329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479097d-7083-4d43-94dc-1760da4270bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:46.538988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479097d-7083-4d43-94dc-1760da4270bb service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:46.539298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479097d-7083-4d43-94dc-1760da4270bb service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:46.539576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479097d-7083-4d43-94dc-1760da4270bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:46.548958 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479097d-7083-4d43-94dc-1760da4270bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:46.550573 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479097d-7083-4d43-94dc-1760da4270bb service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:46.562470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 689/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:48.813598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34be3d-cfc4-4786-8491-fe8f0dcf55ba None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:48.814188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34be3d-cfc4-4786-8491-fe8f0dcf55ba None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:48.814188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34be3d-cfc4-4786-8491-fe8f0dcf55ba None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:48.815591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 691/1380] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:48.821227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb411-4ba9-4501-aa20-2095222e4757 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:48.821725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb411-4ba9-4501-aa20-2095222e4757 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:48.821831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb411-4ba9-4501-aa20-2095222e4757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:48.841984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1f4cb411-4ba9-4501-aa20-2095222e4757 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:48.843538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1f4cb411-4ba9-4501-aa20-2095222e4757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:48.852166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 690/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 03:56:48.898024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34be3d-cfc4-4786-8491-fe8f0dcf55ba None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:48.912508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34be3d-cfc4-4786-8491-fe8f0dcf55ba None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:48.918459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44706c33-24e9-4c87-8afc-e639b57d964d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:48.919106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44706c33-24e9-4c87-8afc-e639b57d964d service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:48.919526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44706c33-24e9-4c87-8afc-e639b57d964d service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:48.919802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44706c33-24e9-4c87-8afc-e639b57d964d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:48.929765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44706c33-24e9-4c87-8afc-e639b57d964d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:48.931636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44706c33-24e9-4c87-8afc-e639b57d964d service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:48.944610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 692/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:51.272187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df414-5161-43fc-825a-8583b4436787 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:51.272778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df414-5161-43fc-825a-8583b4436787 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:51.272778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df414-5161-43fc-825a-8583b4436787 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:51.274107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 691/1383] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:51.279652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc11221-a0fd-40f9-bb33-1de42f5f1baf None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:51.280104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc11221-a0fd-40f9-bb33-1de42f5f1baf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:51.280537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc11221-a0fd-40f9-bb33-1de42f5f1baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:51.300029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0cc11221-a0fd-40f9-bb33-1de42f5f1baf None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:51.301356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0cc11221-a0fd-40f9-bb33-1de42f5f1baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:51.310014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 693/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:51.353118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df414-5161-43fc-825a-8583b4436787 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:51.369719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df414-5161-43fc-825a-8583b4436787 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:51.376314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89f2e-e2c3-4520-9e3b-aa2e1fa3b55b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:51.377121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89f2e-e2c3-4520-9e3b-aa2e1fa3b55b service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:51.377525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89f2e-e2c3-4520-9e3b-aa2e1fa3b55b service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:51.377907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89f2e-e2c3-4520-9e3b-aa2e1fa3b55b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:51.387259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d89f2e-e2c3-4520-9e3b-aa2e1fa3b55b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:51.388991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d89f2e-e2c3-4520-9e3b-aa2e1fa3b55b service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:51.400500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 692/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:53.807263 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b49b5-b2b4-45f0-a7a4-d471b1d50aea None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:53.807800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b49b5-b2b4-45f0-a7a4-d471b1d50aea None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:53.807800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b49b5-b2b4-45f0-a7a4-d471b1d50aea None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:53.815523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b230e4-777e-40ad-a389-9365abf0990d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:53.815523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b230e4-777e-40ad-a389-9365abf0990d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:53.815523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b230e4-777e-40ad-a389-9365abf0990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:53.817530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 694/1386] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:53.837720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-50b230e4-777e-40ad-a389-9365abf0990d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:53.839113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-50b230e4-777e-40ad-a389-9365abf0990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:53.848421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 693/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:53.897177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b49b5-b2b4-45f0-a7a4-d471b1d50aea None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:53.916977 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b49b5-b2b4-45f0-a7a4-d471b1d50aea None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:53.925268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7d65c-8bee-4613-8184-76d8cd4a0d19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:53.926415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7d65c-8bee-4613-8184-76d8cd4a0d19 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:53.927069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7d65c-8bee-4613-8184-76d8cd4a0d19 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:53.927421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7d65c-8bee-4613-8184-76d8cd4a0d19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:53.938465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7d65c-8bee-4613-8184-76d8cd4a0d19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:53.940190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7d65c-8bee-4613-8184-76d8cd4a0d19 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:53.953388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 695/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:56.211759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991d355-3536-407a-a4ea-49484acf1046 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:56.212323 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991d355-3536-407a-a4ea-49484acf1046 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:56.212423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991d355-3536-407a-a4ea-49484acf1046 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:56.214183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 694/1389] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:56.221058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc27c52-ff34-4f3c-a864-b15ac8d4b0ae None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:56.221586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc27c52-ff34-4f3c-a864-b15ac8d4b0ae None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:56.221682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc27c52-ff34-4f3c-a864-b15ac8d4b0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:56.244246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-acc27c52-ff34-4f3c-a864-b15ac8d4b0ae None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:56.245511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-acc27c52-ff34-4f3c-a864-b15ac8d4b0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:56.255732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 696/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:56.309162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991d355-3536-407a-a4ea-49484acf1046 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:56.324924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991d355-3536-407a-a4ea-49484acf1046 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:56.330993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfee7c-a7d4-43ff-966c-e84f657667fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:56.331915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfee7c-a7d4-43ff-966c-e84f657667fb service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:56.332112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfee7c-a7d4-43ff-966c-e84f657667fb service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:56.332351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfee7c-a7d4-43ff-966c-e84f657667fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:56.341954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfee7c-a7d4-43ff-966c-e84f657667fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:56.344157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfee7c-a7d4-43ff-966c-e84f657667fb service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:56.355757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 695/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:56:58.675847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e7f6-de7f-469e-8086-9e6f468bcc47 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:58.676503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e7f6-de7f-469e-8086-9e6f468bcc47 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:58.676503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e7f6-de7f-469e-8086-9e6f468bcc47 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:58.677859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 697/1392] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:56:58.683763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c514ae-9fec-4fdd-bd31-92f710d90450 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:58.684267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c514ae-9fec-4fdd-bd31-92f710d90450 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:58.684621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c514ae-9fec-4fdd-bd31-92f710d90450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:58.708385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c9c514ae-9fec-4fdd-bd31-92f710d90450 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:56:58.710217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c9c514ae-9fec-4fdd-bd31-92f710d90450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:56:58.721219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 696/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:56:58.780938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43e7f6-de7f-469e-8086-9e6f468bcc47 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:56:58.790312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=46 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:56:58.797378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43e7f6-de7f-469e-8086-9e6f468bcc47 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:56:58.803596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7c765-1432-40ce-95af-409c9122e7b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:56:58.813458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7c765-1432-40ce-95af-409c9122e7b0 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:56:58.813458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7c765-1432-40ce-95af-409c9122e7b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:56:58.813458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7c765-1432-40ce-95af-409c9122e7b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:56:58.815353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7c765-1432-40ce-95af-409c9122e7b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:56:58.816980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7c765-1432-40ce-95af-409c9122e7b0 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:56:58.828521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 698/1394] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:01.168685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e61f3c-6da9-42f6-a67c-4666a5f8e326 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:01.169514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e61f3c-6da9-42f6-a67c-4666a5f8e326 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:01.169514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e61f3c-6da9-42f6-a67c-4666a5f8e326 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:01.170984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 697/1395] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:01.178237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe1393-1c08-46c4-9932-5f878ddb69df None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:01.178877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe1393-1c08-46c4-9932-5f878ddb69df None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:01.178877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe1393-1c08-46c4-9932-5f878ddb69df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:01.200856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ebbe1393-1c08-46c4-9932-5f878ddb69df None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:01.203080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ebbe1393-1c08-46c4-9932-5f878ddb69df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:01.212483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 699/1396] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:01.266265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e61f3c-6da9-42f6-a67c-4666a5f8e326 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:01.287571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e61f3c-6da9-42f6-a67c-4666a5f8e326 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:01.294725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ea5f8-85c2-4e78-b25c-426b99c971bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:01.295548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ea5f8-85c2-4e78-b25c-426b99c971bb service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:01.295789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ea5f8-85c2-4e78-b25c-426b99c971bb service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:01.296066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ea5f8-85c2-4e78-b25c-426b99c971bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:01.306663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ea5f8-85c2-4e78-b25c-426b99c971bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:01.308681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ea5f8-85c2-4e78-b25c-426b99c971bb service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:01.321518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 698/1397] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:03.727404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4ad3e-6271-4df8-9294-86a570248063 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:03.728140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4ad3e-6271-4df8-9294-86a570248063 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:03.728140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4ad3e-6271-4df8-9294-86a570248063 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:03.729645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 700/1398] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:57:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:03.736848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57178222-5c90-4d06-aa49-85454dac3ec3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:03.737449 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57178222-5c90-4d06-aa49-85454dac3ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:03.737622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57178222-5c90-4d06-aa49-85454dac3ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:03.763792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-57178222-5c90-4d06-aa49-85454dac3ec3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:03.765413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-57178222-5c90-4d06-aa49-85454dac3ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:03.776366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 699/1399] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:03.839422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4ad3e-6271-4df8-9294-86a570248063 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:03.855201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4ad3e-6271-4df8-9294-86a570248063 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:03.861600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c3971-d014-46b4-bf46-e320c893bf34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:03.862261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c3971-d014-46b4-bf46-e320c893bf34 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:03.862548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c3971-d014-46b4-bf46-e320c893bf34 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:03.862824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c3971-d014-46b4-bf46-e320c893bf34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:03.872603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c3971-d014-46b4-bf46-e320c893bf34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:03.874262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c3971-d014-46b4-bf46-e320c893bf34 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:03.884477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 701/1400] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:05.208929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dce054-127f-499d-af75-cc707e8a4629 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:05.209822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dce054-127f-499d-af75-cc707e8a4629 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:05.209822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dce054-127f-499d-af75-cc707e8a4629 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:05.211598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 700/1401] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:05.219249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e3a16-c68f-4c68-8ac1-ad8a3b2ec0ae None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:05.219786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e3a16-c68f-4c68-8ac1-ad8a3b2ec0ae None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:05.220149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e3a16-c68f-4c68-8ac1-ad8a3b2ec0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:05.239269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4d4e3a16-c68f-4c68-8ac1-ad8a3b2ec0ae None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:05.240491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4d4e3a16-c68f-4c68-8ac1-ad8a3b2ec0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:05.248775 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 702/1402] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:05.306055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be232-303c-4287-b0df-a60e336ae35e None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:05.306343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be232-303c-4287-b0df-a60e336ae35e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:05.306625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be232-303c-4287-b0df-a60e336ae35e None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:05.308027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 701/1403] 10.222.0.20 () {54 vars in 962 bytes} [Fri Jul 1 03:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:57:05.313505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94718032-04f3-4153-8d27-8427ce817229 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:05.313795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94718032-04f3-4153-8d27-8427ce817229 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:05.314073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94718032-04f3-4153-8d27-8427ce817229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:05.364695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-94718032-04f3-4153-8d27-8427ce817229 None None] MFA Rules not processed for user `e8327d9916424843b2a67cbdd856ad37`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:05.365912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-94718032-04f3-4153-8d27-8427ce817229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:05.429285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 703/1404] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Jul 1 03:57:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:05.437262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767be232-303c-4287-b0df-a60e336ae35e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:05.448149 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767be232-303c-4287-b0df-a60e336ae35e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:05.452901 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e033ac-85d9-430f-860d-134285a029a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8327d9916424843b2a67cbdd856ad37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:05.453060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e033ac-85d9-430f-860d-134285a029a5 service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:05.453278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e033ac-85d9-430f-860d-134285a029a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:05.453554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e033ac-85d9-430f-860d-134285a029a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:05.463732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e033ac-85d9-430f-860d-134285a029a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:05.464963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e033ac-85d9-430f-860d-134285a029a5 service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:05.476041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 702/1405] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:06.327412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698361-fb66-4332-b3e4-4024caec81af None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:06.328031 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698361-fb66-4332-b3e4-4024caec81af None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:06.328184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698361-fb66-4332-b3e4-4024caec81af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:06.383929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f1698361-fb66-4332-b3e4-4024caec81af None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:06.385173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f1698361-fb66-4332-b3e4-4024caec81af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:06.453452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 704/1406] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 03:57:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:06.499280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2e07c-b67d-4796-8bec-ee739ce91dc2 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:06.499733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2e07c-b67d-4796-8bec-ee739ce91dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:06.499840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2e07c-b67d-4796-8bec-ee739ce91dc2 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:06.501187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 703/1407] 10.222.0.20 () {54 vars in 942 bytes} [Fri Jul 1 03:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:06.508344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdd5d-0d7f-48b1-ad48-78fe1a2fbdb5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:06.508871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdd5d-0d7f-48b1-ad48-78fe1a2fbdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:06.509289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdd5d-0d7f-48b1-ad48-78fe1a2fbdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:06.529244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:57:06.680033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-83dcdd5d-0d7f-48b1-ad48-78fe1a2fbdb5 None None] MFA Rules not processed for user `1123437cabbd4aa09ca3b1a0996a8bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:06.681250 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-83dcdd5d-0d7f-48b1-ad48-78fe1a2fbdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:06.753809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 705/1408] 10.222.0.20 () {58 vars in 1039 bytes} [Fri Jul 1 03:57:06 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:06.763179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2e07c-b67d-4796-8bec-ee739ce91dc2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:06.776960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2e07c-b67d-4796-8bec-ee739ce91dc2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:06.782449 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc46728-bc52-4b3c-bb84-51f0ed3bed98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:06.783091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc46728-bc52-4b3c-bb84-51f0ed3bed98 service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:06.783406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc46728-bc52-4b3c-bb84-51f0ed3bed98 service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:06.783686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc46728-bc52-4b3c-bb84-51f0ed3bed98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:06.793024 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc46728-bc52-4b3c-bb84-51f0ed3bed98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:06.794605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc46728-bc52-4b3c-bb84-51f0ed3bed98 service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:06.803378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 704/1409] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 03:57:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 03:57:16.750871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=39 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:57:16.800251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=75 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:57:30.839774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47843339-bf00-46e2-8136-3de1fad38887 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:30.840365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47843339-bf00-46e2-8136-3de1fad38887 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:30.840365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47843339-bf00-46e2-8136-3de1fad38887 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:30.841721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 706/1410] 10.222.0.20 () {54 vars in 918 bytes} [Fri Jul 1 03:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:30.846923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ccb0e-5cb6-4c2b-8aea-b29aa0f93990 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:30.847359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ccb0e-5cb6-4c2b-8aea-b29aa0f93990 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:30.847505 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ccb0e-5cb6-4c2b-8aea-b29aa0f93990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:30.866130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cb7ccb0e-5cb6-4c2b-8aea-b29aa0f93990 None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:30.867360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cb7ccb0e-5cb6-4c2b-8aea-b29aa0f93990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:30.876998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 705/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Jul 1 03:57:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:30.891006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b9923-1003-4c46-9f2f-9cfa528f0142 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:30.891329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b9923-1003-4c46-9f2f-9cfa528f0142 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:30.891611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b9923-1003-4c46-9f2f-9cfa528f0142 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:30.892969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 707/1412] 10.222.0.20 () {54 vars in 918 bytes} [Fri Jul 1 03:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:30.897802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e300ee-5424-4b5f-b1c5-87a3aeccb531 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:30.898092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e300ee-5424-4b5f-b1c5-87a3aeccb531 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:30.898373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e300ee-5424-4b5f-b1c5-87a3aeccb531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:30.917708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f4e300ee-5424-4b5f-b1c5-87a3aeccb531 None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:30.918952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f4e300ee-5424-4b5f-b1c5-87a3aeccb531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:30.929283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 706/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Jul 1 03:57:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:33.473465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce21d-8ebe-4ee8-a667-618d9405aac1 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:33.474146 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce21d-8ebe-4ee8-a667-618d9405aac1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:33.474146 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce21d-8ebe-4ee8-a667-618d9405aac1 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:33.475335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:33.481107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dac21-2de0-4918-a0f2-52f78f6d4289 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:33.481769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dac21-2de0-4918-a0f2-52f78f6d4289 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:33.481769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dac21-2de0-4918-a0f2-52f78f6d4289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:33.501359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ec7dac21-2de0-4918-a0f2-52f78f6d4289 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:33.503210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ec7dac21-2de0-4918-a0f2-52f78f6d4289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:33.510801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:33.556372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce21d-8ebe-4ee8-a667-618d9405aac1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:33.571648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce21d-8ebe-4ee8-a667-618d9405aac1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:33.577253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8777ee-c8df-4fd1-a0b4-874d949892a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:33.578000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8777ee-c8df-4fd1-a0b4-874d949892a2 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:33.578282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8777ee-c8df-4fd1-a0b4-874d949892a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:33.578558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8777ee-c8df-4fd1-a0b4-874d949892a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:33.587524 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8777ee-c8df-4fd1-a0b4-874d949892a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:33.590962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8777ee-c8df-4fd1-a0b4-874d949892a2 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:33.599853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 709/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 03:57:34.509634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ebf7b-e21d-477e-ac37-e8f210eef4d9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:34.510248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ebf7b-e21d-477e-ac37-e8f210eef4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:34.510248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ebf7b-e21d-477e-ac37-e8f210eef4d9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:34.511659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 03:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:34.517531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae994609-0d9d-4c99-b31e-b3d890d9074c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:34.517977 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae994609-0d9d-4c99-b31e-b3d890d9074c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:34.518230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae994609-0d9d-4c99-b31e-b3d890d9074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:34.536744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ae994609-0d9d-4c99-b31e-b3d890d9074c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:34.537981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ae994609-0d9d-4c99-b31e-b3d890d9074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:34.545982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 03:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:34.696904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ebf7b-e21d-477e-ac37-e8f210eef4d9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:34.714059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ebf7b-e21d-477e-ac37-e8f210eef4d9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:34.717007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f596b48-4d0d-4dd2-bfd8-1660da098a26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:34.717641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596b48-4d0d-4dd2-bfd8-1660da098a26 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:34.717922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596b48-4d0d-4dd2-bfd8-1660da098a26 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:34.718195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596b48-4d0d-4dd2-bfd8-1660da098a26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:34.726766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f596b48-4d0d-4dd2-bfd8-1660da098a26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:34.728369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f596b48-4d0d-4dd2-bfd8-1660da098a26 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:34.738513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 709/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:35.639608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a119bce-4039-4864-afe7-ad804cc90184 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:35.640174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a119bce-4039-4864-afe7-ad804cc90184 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:35.640174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a119bce-4039-4864-afe7-ad804cc90184 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:35.641528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 03:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:35.647020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000edc8-6baa-43f0-8b43-74529bf318bb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:35.647587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000edc8-6baa-43f0-8b43-74529bf318bb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:35.647699 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000edc8-6baa-43f0-8b43-74529bf318bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:35.667463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d000edc8-6baa-43f0-8b43-74529bf318bb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:35.668712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d000edc8-6baa-43f0-8b43-74529bf318bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:35.677165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 03:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:35.828097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a119bce-4039-4864-afe7-ad804cc90184 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:35.842313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a119bce-4039-4864-afe7-ad804cc90184 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:35.847718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999cce7-61cc-4576-ac76-52f5f39c3a80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:35.848346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cce7-61cc-4576-ac76-52f5f39c3a80 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:35.848621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cce7-61cc-4576-ac76-52f5f39c3a80 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:35.848889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cce7-61cc-4576-ac76-52f5f39c3a80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:35.857093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cce7-61cc-4576-ac76-52f5f39c3a80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:35.858765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cce7-61cc-4576-ac76-52f5f39c3a80 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:35.868610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 712/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:36.726155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c197-c491-43e5-8720-7ae7bbfbdfa6 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:36.726714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c197-c491-43e5-8720-7ae7bbfbdfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:36.726714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c197-c491-43e5-8720-7ae7bbfbdfa6 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:36.728097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 03:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:36.733877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7d74-76d7-4731-bc45-ecd1add6e61d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:36.734327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7d74-76d7-4731-bc45-ecd1add6e61d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:36.734432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7d74-76d7-4731-bc45-ecd1add6e61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:36.752416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0cae7d74-76d7-4731-bc45-ecd1add6e61d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:36.753633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0cae7d74-76d7-4731-bc45-ecd1add6e61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:36.761399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 03:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:36.921309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89c197-c491-43e5-8720-7ae7bbfbdfa6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:36.936323 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89c197-c491-43e5-8720-7ae7bbfbdfa6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:36.941669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752a14a-4962-4072-9313-0621fb600183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:36.942304 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a14a-4962-4072-9313-0621fb600183 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:36.942585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a14a-4962-4072-9313-0621fb600183 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:36.942856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a14a-4962-4072-9313-0621fb600183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:36.951639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752a14a-4962-4072-9313-0621fb600183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:36.953204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752a14a-4962-4072-9313-0621fb600183 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:36.963019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 712/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:37.859288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e71d3-08dc-4b45-be84-c76cc57d7da5 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:37.859885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e71d3-08dc-4b45-be84-c76cc57d7da5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:37.859885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e71d3-08dc-4b45-be84-c76cc57d7da5 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:37.861211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 03:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:37.866382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059097d1-a990-47a8-8e9e-2afcc99f59d3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:37.866924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059097d1-a990-47a8-8e9e-2afcc99f59d3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:37.866924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059097d1-a990-47a8-8e9e-2afcc99f59d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:37.886713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-059097d1-a990-47a8-8e9e-2afcc99f59d3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:37.888015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-059097d1-a990-47a8-8e9e-2afcc99f59d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:37.896172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 03:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:38.047302 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e71d3-08dc-4b45-be84-c76cc57d7da5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:38.061794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e71d3-08dc-4b45-be84-c76cc57d7da5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:38.067191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e5d69-1fdf-4981-9e90-b3c79e077cde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:38.067856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e5d69-1fdf-4981-9e90-b3c79e077cde service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:38.068139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e5d69-1fdf-4981-9e90-b3c79e077cde service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:38.068417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e5d69-1fdf-4981-9e90-b3c79e077cde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:38.076554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e5d69-1fdf-4981-9e90-b3c79e077cde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:38.078122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e5d69-1fdf-4981-9e90-b3c79e077cde service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:38.087513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 715/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:38.957063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bfc8-189f-4421-b2e6-48f402e8eac3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:38.957611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bfc8-189f-4421-b2e6-48f402e8eac3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:38.957611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bfc8-189f-4421-b2e6-48f402e8eac3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:38.958910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 03:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:57:38.966739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b80a0-14aa-4e79-a292-b37cd982e7aa None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:38.967336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b80a0-14aa-4e79-a292-b37cd982e7aa None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:38.967336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b80a0-14aa-4e79-a292-b37cd982e7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:38.986008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6b4b80a0-14aa-4e79-a292-b37cd982e7aa None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:38.987272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6b4b80a0-14aa-4e79-a292-b37cd982e7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:38.995276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 03:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:39.145876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572bfc8-189f-4421-b2e6-48f402e8eac3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:39.160704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572bfc8-189f-4421-b2e6-48f402e8eac3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:39.166826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbba284-9a4d-49ec-bad6-bf663e6ae7da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:39.167502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba284-9a4d-49ec-bad6-bf663e6ae7da service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:39.167782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba284-9a4d-49ec-bad6-bf663e6ae7da service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:39.168052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba284-9a4d-49ec-bad6-bf663e6ae7da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:39.176607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbba284-9a4d-49ec-bad6-bf663e6ae7da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:39.178172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbba284-9a4d-49ec-bad6-bf663e6ae7da service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:39.188119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 715/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:40.089597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3389c2-57a4-4109-83f3-e661ee1a49da None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:40.090163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3389c2-57a4-4109-83f3-e661ee1a49da None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:40.090163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3389c2-57a4-4109-83f3-e661ee1a49da None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:40.091728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 03:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:40.097673 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d64d7a-74c4-4cf9-b17b-738810000bfe None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:40.098288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d64d7a-74c4-4cf9-b17b-738810000bfe None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:40.098288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d64d7a-74c4-4cf9-b17b-738810000bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:40.118481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c3d64d7a-74c4-4cf9-b17b-738810000bfe None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:40.119791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c3d64d7a-74c4-4cf9-b17b-738810000bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:40.128654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 03:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:40.281028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3389c2-57a4-4109-83f3-e661ee1a49da None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:40.299868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3389c2-57a4-4109-83f3-e661ee1a49da None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:40.305986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ffa8f-fc90-46bf-acf3-badd87792915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:40.306673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ffa8f-fc90-46bf-acf3-badd87792915 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:40.306971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ffa8f-fc90-46bf-acf3-badd87792915 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:40.307279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ffa8f-fc90-46bf-acf3-badd87792915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:40.316789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ffa8f-fc90-46bf-acf3-badd87792915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:40.330423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ffa8f-fc90-46bf-acf3-badd87792915 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:40.330423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 718/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:41.229840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe5591-c49c-4cb2-9718-bb0fc1e7a117 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:41.230451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe5591-c49c-4cb2-9718-bb0fc1e7a117 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:41.230451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe5591-c49c-4cb2-9718-bb0fc1e7a117 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:41.232511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 03:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:41.238070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6f1e1-d902-49a1-8472-3a623f78324e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:41.238623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6f1e1-d902-49a1-8472-3a623f78324e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:41.238623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6f1e1-d902-49a1-8472-3a623f78324e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:41.257267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a0d6f1e1-d902-49a1-8472-3a623f78324e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:41.258466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a0d6f1e1-d902-49a1-8472-3a623f78324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:41.266335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 03:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:41.417956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe5591-c49c-4cb2-9718-bb0fc1e7a117 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:41.432139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe5591-c49c-4cb2-9718-bb0fc1e7a117 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:41.437710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4beeb1-f1e7-46f3-a688-f8ea7538893f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:41.438336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4beeb1-f1e7-46f3-a688-f8ea7538893f service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:41.438613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4beeb1-f1e7-46f3-a688-f8ea7538893f service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:41.438880 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4beeb1-f1e7-46f3-a688-f8ea7538893f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:41.446983 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4beeb1-f1e7-46f3-a688-f8ea7538893f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:41.448575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4beeb1-f1e7-46f3-a688-f8ea7538893f service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:41.458851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 718/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:42.303902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1277a-b7a1-4d04-8335-696dec696fe6 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:42.304506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1277a-b7a1-4d04-8335-696dec696fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:42.304506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1277a-b7a1-4d04-8335-696dec696fe6 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:42.305841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 03:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:42.311208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a80a1-f3cf-476e-9689-2413cbaa147e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:42.311683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a80a1-f3cf-476e-9689-2413cbaa147e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:42.311788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a80a1-f3cf-476e-9689-2413cbaa147e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:42.331528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6e8a80a1-f3cf-476e-9689-2413cbaa147e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:42.332812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6e8a80a1-f3cf-476e-9689-2413cbaa147e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:42.341730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 03:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 03:57:42.492312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1277a-b7a1-4d04-8335-696dec696fe6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:42.507470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1277a-b7a1-4d04-8335-696dec696fe6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:42.513896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1fc4f-1e32-4cf7-bc37-82a0b8bfcb92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:42.514544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1fc4f-1e32-4cf7-bc37-82a0b8bfcb92 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:42.514856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1fc4f-1e32-4cf7-bc37-82a0b8bfcb92 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:42.515089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1fc4f-1e32-4cf7-bc37-82a0b8bfcb92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:42.524174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1fc4f-1e32-4cf7-bc37-82a0b8bfcb92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:42.526043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1fc4f-1e32-4cf7-bc37-82a0b8bfcb92 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:42.535697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 721/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:44.491581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28d27e-46b5-420d-a541-46988277ef69 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:44.492220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28d27e-46b5-420d-a541-46988277ef69 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:44.492220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28d27e-46b5-420d-a541-46988277ef69 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:44.493589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:57:44.500081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51005f-a066-4ad5-89d8-47126661d7c7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:44.500615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51005f-a066-4ad5-89d8-47126661d7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:44.500615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51005f-a066-4ad5-89d8-47126661d7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:44.518680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0e51005f-a066-4ad5-89d8-47126661d7c7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:44.519923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0e51005f-a066-4ad5-89d8-47126661d7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:44.527850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:44.567349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28d27e-46b5-420d-a541-46988277ef69 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:44.578010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28d27e-46b5-420d-a541-46988277ef69 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:44.581627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1d010-6f6f-4781-8212-b1794367bd75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:44.582252 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d010-6f6f-4781-8212-b1794367bd75 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:44.582536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d010-6f6f-4781-8212-b1794367bd75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:44.582807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d010-6f6f-4781-8212-b1794367bd75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:44.592182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1d010-6f6f-4781-8212-b1794367bd75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:44.593644 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1d010-6f6f-4781-8212-b1794367bd75 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:44.603263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 721/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:46.782666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4743-e0ae-4f8b-b5bd-3e3b1f94d8c6 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:46.783280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4743-e0ae-4f8b-b5bd-3e3b1f94d8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:46.783280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4743-e0ae-4f8b-b5bd-3e3b1f94d8c6 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:46.784634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:46.790636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ba741-1d89-496c-85fc-64b24a5a309a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:46.791185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ba741-1d89-496c-85fc-64b24a5a309a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:46.791185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ba741-1d89-496c-85fc-64b24a5a309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:46.810862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-625ba741-1d89-496c-85fc-64b24a5a309a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:46.812180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-625ba741-1d89-496c-85fc-64b24a5a309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:46.820334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:46.858638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a4743-e0ae-4f8b-b5bd-3e3b1f94d8c6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:46.869697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a4743-e0ae-4f8b-b5bd-3e3b1f94d8c6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:46.873779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b255a-ca4c-44f9-9cc2-ba3116861037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:46.874427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b255a-ca4c-44f9-9cc2-ba3116861037 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:46.874710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b255a-ca4c-44f9-9cc2-ba3116861037 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:46.874989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b255a-ca4c-44f9-9cc2-ba3116861037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:46.884454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b255a-ca4c-44f9-9cc2-ba3116861037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:46.885855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b255a-ca4c-44f9-9cc2-ba3116861037 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:46.895406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 724/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:49.424889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03420ce-0a3f-4e3c-855c-5729fdcf21e0 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:49.425521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03420ce-0a3f-4e3c-855c-5729fdcf21e0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:49.425521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03420ce-0a3f-4e3c-855c-5729fdcf21e0 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:49.426813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:57:49.432631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d6f07-a523-4227-a7af-a55e2bad6e1b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:49.433205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d6f07-a523-4227-a7af-a55e2bad6e1b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:49.433205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d6f07-a523-4227-a7af-a55e2bad6e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:49.457529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6b8d6f07-a523-4227-a7af-a55e2bad6e1b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:49.458739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6b8d6f07-a523-4227-a7af-a55e2bad6e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:49.467395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:49.507422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03420ce-0a3f-4e3c-855c-5729fdcf21e0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:49.509153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c03420ce-0a3f-4e3c-855c-5729fdcf21e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:49.523041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03420ce-0a3f-4e3c-855c-5729fdcf21e0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:49.526642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4974a-2c6a-4999-8b51-fd4eed1f6d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:49.527319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4974a-2c6a-4999-8b51-fd4eed1f6d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:49.527603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4974a-2c6a-4999-8b51-fd4eed1f6d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:49.528400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4974a-2c6a-4999-8b51-fd4eed1f6d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:49.538182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4974a-2c6a-4999-8b51-fd4eed1f6d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:49.539744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4974a-2c6a-4999-8b51-fd4eed1f6d7c service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:49.550536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 724/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:51.736537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638062-d7d9-44da-9629-641100cba94d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:51.737077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638062-d7d9-44da-9629-641100cba94d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:51.737077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638062-d7d9-44da-9629-641100cba94d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:51.738464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:51.743912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5053545-f7f5-41b4-8490-12fe0fb6a7ff None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:51.744334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5053545-f7f5-41b4-8490-12fe0fb6a7ff None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:51.744485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5053545-f7f5-41b4-8490-12fe0fb6a7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:51.762100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d5053545-f7f5-41b4-8490-12fe0fb6a7ff None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:51.763332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d5053545-f7f5-41b4-8490-12fe0fb6a7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:51.771191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:51.810802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28638062-d7d9-44da-9629-641100cba94d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:51.821729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28638062-d7d9-44da-9629-641100cba94d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:51.825253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dc03b-b17c-4cdb-bc24-b7db2d9940ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:51.825884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dc03b-b17c-4cdb-bc24-b7db2d9940ed service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:51.826175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dc03b-b17c-4cdb-bc24-b7db2d9940ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:51.826454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dc03b-b17c-4cdb-bc24-b7db2d9940ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:51.835928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4dc03b-b17c-4cdb-bc24-b7db2d9940ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:51.837340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4dc03b-b17c-4cdb-bc24-b7db2d9940ed service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:51.847494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 727/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:54.023205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f422a-4573-4e44-bed1-b4cf1c4d2952 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:54.023811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f422a-4573-4e44-bed1-b4cf1c4d2952 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:54.023919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f422a-4573-4e44-bed1-b4cf1c4d2952 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:54.025384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:54.030892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abafb2-6090-45f3-a522-99de5a98838c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:54.031407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abafb2-6090-45f3-a522-99de5a98838c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:54.031518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abafb2-6090-45f3-a522-99de5a98838c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:54.049013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-35abafb2-6090-45f3-a522-99de5a98838c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:54.050213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-35abafb2-6090-45f3-a522-99de5a98838c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:54.058160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:54.096788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f422a-4573-4e44-bed1-b4cf1c4d2952 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:54.108163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f422a-4573-4e44-bed1-b4cf1c4d2952 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:54.111733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246d9f0-df3a-49e4-a0c3-88b4f58d49bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:54.112377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246d9f0-df3a-49e4-a0c3-88b4f58d49bb service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:54.112656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246d9f0-df3a-49e4-a0c3-88b4f58d49bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:54.112931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246d9f0-df3a-49e4-a0c3-88b4f58d49bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:54.122855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246d9f0-df3a-49e4-a0c3-88b4f58d49bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:54.124324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246d9f0-df3a-49e4-a0c3-88b4f58d49bb service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:54.133905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 727/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:56.225139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdb73d-b658-4d02-929c-400b6a162501 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:56.225719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdb73d-b658-4d02-929c-400b6a162501 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:56.225719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdb73d-b658-4d02-929c-400b6a162501 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:56.227041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:56.232637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cceb0a-1709-4ed1-bf84-a771806b5689 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:56.233106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cceb0a-1709-4ed1-bf84-a771806b5689 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:56.233203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cceb0a-1709-4ed1-bf84-a771806b5689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:56.251820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c6cceb0a-1709-4ed1-bf84-a771806b5689 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:56.253039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c6cceb0a-1709-4ed1-bf84-a771806b5689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:56.261219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:56.299335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdb73d-b658-4d02-929c-400b6a162501 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:56.310966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdb73d-b658-4d02-929c-400b6a162501 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:56.314574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bcab2-f39f-4cd9-a7a5-aaffa021b33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:56.315241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bcab2-f39f-4cd9-a7a5-aaffa021b33e service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:56.315521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bcab2-f39f-4cd9-a7a5-aaffa021b33e service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:56.315795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bcab2-f39f-4cd9-a7a5-aaffa021b33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:56.325526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bcab2-f39f-4cd9-a7a5-aaffa021b33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:56.330402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bcab2-f39f-4cd9-a7a5-aaffa021b33e service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:56.336903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 730/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:57:58.478106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5499-5004-4b15-b2ea-5260b62b62e4 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:58.478654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5499-5004-4b15-b2ea-5260b62b62e4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:58.478654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5499-5004-4b15-b2ea-5260b62b62e4 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:58.480097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:57:58.485628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17e7ad-aeaf-4035-bb18-077543f2949e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:58.486155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17e7ad-aeaf-4035-bb18-077543f2949e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:58.486262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17e7ad-aeaf-4035-bb18-077543f2949e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:58.492214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=70 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:57:58.504671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dd17e7ad-aeaf-4035-bb18-077543f2949e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:57:58.509662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dd17e7ad-aeaf-4035-bb18-077543f2949e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:57:58.514602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:57:58.551948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f5499-5004-4b15-b2ea-5260b62b62e4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:57:58.557558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=73 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:57:58.564239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f5499-5004-4b15-b2ea-5260b62b62e4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:57:58.568292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b610b6-20d8-4260-9311-3fd722b3d8ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:57:58.569006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b610b6-20d8-4260-9311-3fd722b3d8ad service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:57:58.569293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b610b6-20d8-4260-9311-3fd722b3d8ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:57:58.569568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b610b6-20d8-4260-9311-3fd722b3d8ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:57:58.579626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b610b6-20d8-4260-9311-3fd722b3d8ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:57:58.581095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b610b6-20d8-4260-9311-3fd722b3d8ad service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:57:58.591672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 730/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:00.699551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b80a-d1ae-496f-ba01-c61ffd1d170d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:00.700129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b80a-d1ae-496f-ba01-c61ffd1d170d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:00.700129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b80a-d1ae-496f-ba01-c61ffd1d170d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:00.701473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:00.707001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bf41a-4acf-4d16-8fb3-bff4f7553470 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:00.707496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bf41a-4acf-4d16-8fb3-bff4f7553470 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:00.707616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bf41a-4acf-4d16-8fb3-bff4f7553470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:00.725784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7f7bf41a-4acf-4d16-8fb3-bff4f7553470 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:00.727043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7f7bf41a-4acf-4d16-8fb3-bff4f7553470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:00.735430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:00.772612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1b80a-d1ae-496f-ba01-c61ffd1d170d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:00.784067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1b80a-d1ae-496f-ba01-c61ffd1d170d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:00.787551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4e616-21aa-4935-9c46-6df1293b41be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:00.788187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e616-21aa-4935-9c46-6df1293b41be service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:00.788466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e616-21aa-4935-9c46-6df1293b41be service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:00.788741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e616-21aa-4935-9c46-6df1293b41be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:00.798938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4e616-21aa-4935-9c46-6df1293b41be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:00.800469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4e616-21aa-4935-9c46-6df1293b41be service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:00.810338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 733/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:02.885847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a6de0-fde3-4f4d-98d6-33856875108d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:02.886403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a6de0-fde3-4f4d-98d6-33856875108d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:02.886403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a6de0-fde3-4f4d-98d6-33856875108d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:02.887847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:02.893299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96153ba-bf69-4eda-9336-39e43d463ec0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:02.893734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96153ba-bf69-4eda-9336-39e43d463ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:02.893921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96153ba-bf69-4eda-9336-39e43d463ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:02.911920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d96153ba-bf69-4eda-9336-39e43d463ec0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:02.913160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d96153ba-bf69-4eda-9336-39e43d463ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:02.921575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:02.957949 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a6de0-fde3-4f4d-98d6-33856875108d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:02.969283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a6de0-fde3-4f4d-98d6-33856875108d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:02.972844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150e8b7-7f0b-450f-8f10-e960f3aea4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:02.973485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150e8b7-7f0b-450f-8f10-e960f3aea4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:02.973771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150e8b7-7f0b-450f-8f10-e960f3aea4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:02.974048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150e8b7-7f0b-450f-8f10-e960f3aea4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:02.984051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150e8b7-7f0b-450f-8f10-e960f3aea4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:02.985484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150e8b7-7f0b-450f-8f10-e960f3aea4f1 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:02.995161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 733/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:05.066885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a164b-1a61-49b6-bd5d-de98628c099e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:05.067518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a164b-1a61-49b6-bd5d-de98628c099e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:05.067518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a164b-1a61-49b6-bd5d-de98628c099e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:05.068877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 03:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:05.075501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2dd35-887f-4ea4-b16b-e0669223ac31 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:05.076077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2dd35-887f-4ea4-b16b-e0669223ac31 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:05.076077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2dd35-887f-4ea4-b16b-e0669223ac31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:05.093714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-adf2dd35-887f-4ea4-b16b-e0669223ac31 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:05.094933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-adf2dd35-887f-4ea4-b16b-e0669223ac31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:05.102896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 03:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:05.140743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a164b-1a61-49b6-bd5d-de98628c099e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:05.151631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a164b-1a61-49b6-bd5d-de98628c099e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:05.155276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09745951-7863-4fa3-a643-53c696a0f615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:05.155924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09745951-7863-4fa3-a643-53c696a0f615 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:05.156198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09745951-7863-4fa3-a643-53c696a0f615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:05.156470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09745951-7863-4fa3-a643-53c696a0f615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:05.165691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09745951-7863-4fa3-a643-53c696a0f615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:05.167199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09745951-7863-4fa3-a643-53c696a0f615 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:05.177026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 736/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:15.088721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=17 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:58:15.172798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=17 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:58:25.307142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05abc9-7fca-4b27-99aa-2f444c33b32d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:25.308320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05abc9-7fca-4b27-99aa-2f444c33b32d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:25.308320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05abc9-7fca-4b27-99aa-2f444c33b32d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:25.309911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 735/1471] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:25.315590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294a6b8-7701-4ca5-875a-05aa3c40add7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:25.316040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294a6b8-7701-4ca5-875a-05aa3c40add7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:25.316137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294a6b8-7701-4ca5-875a-05aa3c40add7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:25.334620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d294a6b8-7701-4ca5-875a-05aa3c40add7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:25.335845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d294a6b8-7701-4ca5-875a-05aa3c40add7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:25.344000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 737/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:25.393202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dcab0-7a1d-4f57-9fd5-4df6bd5f0743 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:25.393501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dcab0-7a1d-4f57-9fd5-4df6bd5f0743 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:25.393774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dcab0-7a1d-4f57-9fd5-4df6bd5f0743 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:25.395133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 736/1473] 10.222.0.20 () {54 vars in 962 bytes} [Fri Jul 1 03:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:25.400088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0cd2-2785-4b82-9a51-ce0c028b5401 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:25.400391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0cd2-2785-4b82-9a51-ce0c028b5401 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:25.400694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0cd2-2785-4b82-9a51-ce0c028b5401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:25.419874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3a0f0cd2-2785-4b82-9a51-ce0c028b5401 None None] MFA Rules not processed for user `e8327d9916424843b2a67cbdd856ad37`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:25.421100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3a0f0cd2-2785-4b82-9a51-ce0c028b5401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:25.427754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 738/1474] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Jul 1 03:58:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:25.436126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dcab0-7a1d-4f57-9fd5-4df6bd5f0743 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:25.446815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dcab0-7a1d-4f57-9fd5-4df6bd5f0743 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:25.451207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc6b5e-c88d-489f-a9b0-b4f74f9d05d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8327d9916424843b2a67cbdd856ad37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:25.451873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc6b5e-c88d-489f-a9b0-b4f74f9d05d0 service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:25.452161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc6b5e-c88d-489f-a9b0-b4f74f9d05d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:25.452432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc6b5e-c88d-489f-a9b0-b4f74f9d05d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:25.462320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc6b5e-c88d-489f-a9b0-b4f74f9d05d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:25.463788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc6b5e-c88d-489f-a9b0-b4f74f9d05d0 service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:25.473508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 737/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:27.641159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accebf09-c4bb-4941-8ee2-b7f3b7047a81 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:27.641730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accebf09-c4bb-4941-8ee2-b7f3b7047a81 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:27.641730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accebf09-c4bb-4941-8ee2-b7f3b7047a81 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:27.643084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 739/1476] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:27.648594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634728fa-7300-43f2-b55e-464ff88943e0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:27.649057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634728fa-7300-43f2-b55e-464ff88943e0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:27.649158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634728fa-7300-43f2-b55e-464ff88943e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:27.667745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-634728fa-7300-43f2-b55e-464ff88943e0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:27.668989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-634728fa-7300-43f2-b55e-464ff88943e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:27.676972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 738/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:27.702418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accebf09-c4bb-4941-8ee2-b7f3b7047a81 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:27.715546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accebf09-c4bb-4941-8ee2-b7f3b7047a81 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:27.720056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea90de-5412-4dba-a0dc-6887e7db874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:27.720755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea90de-5412-4dba-a0dc-6887e7db874a admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:27.721049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea90de-5412-4dba-a0dc-6887e7db874a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:27.721325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea90de-5412-4dba-a0dc-6887e7db874a admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:27.722197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea90de-5412-4dba-a0dc-6887e7db874a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:27.723878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-0bea90de-5412-4dba-a0dc-6887e7db874a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 03:58:27.723878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 01 03:58:27.723878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: Jul 01 03:58:27.725010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea90de-5412-4dba-a0dc-6887e7db874a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:27.744335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 740/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 1 03:58:27 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 03:58:29.834376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284cf1e-2eb1-497a-a894-08c8ade7c827 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:29.834951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284cf1e-2eb1-497a-a894-08c8ade7c827 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:29.834951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284cf1e-2eb1-497a-a894-08c8ade7c827 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:29.836327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 739/1479] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:29.842032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847eb2-608e-417d-97b2-431cbd26c1ad None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:29.842477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847eb2-608e-417d-97b2-431cbd26c1ad None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:29.842588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847eb2-608e-417d-97b2-431cbd26c1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:29.860841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-cd847eb2-608e-417d-97b2-431cbd26c1ad None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:29.862059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cd847eb2-608e-417d-97b2-431cbd26c1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:29.870474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 741/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:29.914253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284cf1e-2eb1-497a-a894-08c8ade7c827 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:29.928241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284cf1e-2eb1-497a-a894-08c8ade7c827 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:29.934601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b2f8b-1558-4e7c-83a8-c03362aa4801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:29.935575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b2f8b-1558-4e7c-83a8-c03362aa4801 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:29.935854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b2f8b-1558-4e7c-83a8-c03362aa4801 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:29.936123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b2f8b-1558-4e7c-83a8-c03362aa4801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:29.944577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b2f8b-1558-4e7c-83a8-c03362aa4801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:29.946166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b2f8b-1558-4e7c-83a8-c03362aa4801 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:29.957424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 740/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:32.202148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5253-181e-48f8-a369-98cb76a86024 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:32.202772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5253-181e-48f8-a369-98cb76a86024 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:32.202772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5253-181e-48f8-a369-98cb76a86024 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:32.204148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 742/1482] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 03:58:32.209542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88673f4c-fe7c-4f76-a4bd-1ce6fa8ce6c5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:32.210012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88673f4c-fe7c-4f76-a4bd-1ce6fa8ce6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:32.210214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88673f4c-fe7c-4f76-a4bd-1ce6fa8ce6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:32.228745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-88673f4c-fe7c-4f76-a4bd-1ce6fa8ce6c5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:32.230537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-88673f4c-fe7c-4f76-a4bd-1ce6fa8ce6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:32.238857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 741/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:32.276450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c5253-181e-48f8-a369-98cb76a86024 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:32.288350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c5253-181e-48f8-a369-98cb76a86024 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:32.292600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c6b68-9317-4b17-af17-1a575a08fb16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:32.293262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c6b68-9317-4b17-af17-1a575a08fb16 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:32.293535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c6b68-9317-4b17-af17-1a575a08fb16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:32.293805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c6b68-9317-4b17-af17-1a575a08fb16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:32.303734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c6b68-9317-4b17-af17-1a575a08fb16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:32.305181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c6b68-9317-4b17-af17-1a575a08fb16 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:32.316735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 743/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:34.422036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cdf04-b113-4cbe-b909-8d97b660e982 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:34.422629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cdf04-b113-4cbe-b909-8d97b660e982 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:34.422629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cdf04-b113-4cbe-b909-8d97b660e982 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:34.424011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 742/1485] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:34.429324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5625f36-fcdc-4c26-8dcc-9c969cbf63fc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:34.429807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5625f36-fcdc-4c26-8dcc-9c969cbf63fc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:34.430040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5625f36-fcdc-4c26-8dcc-9c969cbf63fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:34.449669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d5625f36-fcdc-4c26-8dcc-9c969cbf63fc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:34.450885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d5625f36-fcdc-4c26-8dcc-9c969cbf63fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:34.458830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 744/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:34.497684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cdf04-b113-4cbe-b909-8d97b660e982 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:34.508490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cdf04-b113-4cbe-b909-8d97b660e982 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:34.512749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94091723-28e5-4778-af30-2f910f814e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:34.513749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94091723-28e5-4778-af30-2f910f814e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:34.513749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94091723-28e5-4778-af30-2f910f814e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:34.513921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94091723-28e5-4778-af30-2f910f814e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:34.525396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94091723-28e5-4778-af30-2f910f814e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:34.525525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94091723-28e5-4778-af30-2f910f814e54 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:34.535534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 743/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:36.696280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb0a0a-51f2-43e1-b2e1-e6cf47d6829d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:36.696827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb0a0a-51f2-43e1-b2e1-e6cf47d6829d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:36.696827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb0a0a-51f2-43e1-b2e1-e6cf47d6829d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:36.698226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 745/1488] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:36.704030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee71a6-513f-442b-b447-e29d4401f33b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:36.704534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee71a6-513f-442b-b447-e29d4401f33b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:36.705013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee71a6-513f-442b-b447-e29d4401f33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:36.725832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-55ee71a6-513f-442b-b447-e29d4401f33b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:36.727046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-55ee71a6-513f-442b-b447-e29d4401f33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:36.735732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 744/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:36.776631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb0a0a-51f2-43e1-b2e1-e6cf47d6829d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:36.788652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb0a0a-51f2-43e1-b2e1-e6cf47d6829d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:36.793463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c1810-484e-4000-b6a8-8a987dcf3855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:36.794300 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c1810-484e-4000-b6a8-8a987dcf3855 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:36.794605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c1810-484e-4000-b6a8-8a987dcf3855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:36.794877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c1810-484e-4000-b6a8-8a987dcf3855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:36.805132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c1810-484e-4000-b6a8-8a987dcf3855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:36.806571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c1810-484e-4000-b6a8-8a987dcf3855 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:36.816663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 746/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:39.386268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b0967-8c6d-4254-8237-477267ad6fbb None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:39.386833 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b0967-8c6d-4254-8237-477267ad6fbb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:39.386833 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b0967-8c6d-4254-8237-477267ad6fbb None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:39.388264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 745/1491] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 1 03:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 03:58:39.393954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd1c1a-db82-4aba-8e63-6925b51072df None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:39.394388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd1c1a-db82-4aba-8e63-6925b51072df None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:39.394534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd1c1a-db82-4aba-8e63-6925b51072df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:39.412770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-58bd1c1a-db82-4aba-8e63-6925b51072df None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 03:58:39.413961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-58bd1c1a-db82-4aba-8e63-6925b51072df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 03:58:39.421875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 747/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 1 03:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 03:58:39.460974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b0967-8c6d-4254-8237-477267ad6fbb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 03:58:39.472605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b0967-8c6d-4254-8237-477267ad6fbb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 03:58:39.476158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95200aa-4aed-4e57-8d89-3371b0be22b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 03:58:39.476797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95200aa-4aed-4e57-8d89-3371b0be22b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 03:58:39.477076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95200aa-4aed-4e57-8d89-3371b0be22b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 03:58:39.477346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95200aa-4aed-4e57-8d89-3371b0be22b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 03:58:39.486868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95200aa-4aed-4e57-8d89-3371b0be22b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 03:58:39.488324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95200aa-4aed-4e57-8d89-3371b0be22b8 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 03:58:39.498269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 746/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 03:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 03:58:49.407766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=33 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 03:58:49.493481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=33 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:00:47.225114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1f311-7cd2-4eb0-821e-1f67a6ca95c6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.225738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1f311-7cd2-4eb0-821e-1f67a6ca95c6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.225738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1f311-7cd2-4eb0-821e-1f67a6ca95c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.243960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-27b1f311-7cd2-4eb0-821e-1f67a6ca95c6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:00:47.245206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-27b1f311-7cd2-4eb0-821e-1f67a6ca95c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:00:47.253105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 748/1494] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:00:47.261662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a2cbe-fdf9-4d30-b234-488276fd1694 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.262300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a2cbe-fdf9-4d30-b234-488276fd1694 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.262300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a2cbe-fdf9-4d30-b234-488276fd1694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.280651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4a2a2cbe-fdf9-4d30-b234-488276fd1694 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:00:47.281850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4a2a2cbe-fdf9-4d30-b234-488276fd1694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:00:47.289694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 747/1495] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:00:47.294432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1f311-7cd2-4eb0-821e-1f67a6ca95c6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:47.304360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1f311-7cd2-4eb0-821e-1f67a6ca95c6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:47.309337 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:47.310000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.310541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.310816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.311588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:00:47.311894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:47.312915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:00:47.312915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 01 04:00:47.312915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: Jul 01 04:00:47.313791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:47.326354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 749/1496] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:00:47.341076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a2cbe-fdf9-4d30-b234-488276fd1694 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:47.350226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a2cbe-fdf9-4d30-b234-488276fd1694 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:47.354900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:47.355583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.355863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.356322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.356823 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:47.358575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:00:47.358575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 01 04:00:47.358575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: Jul 01 04:00:47.359794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:47.463273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 748/1497] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:00:47 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:00:47.467922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:47.478387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf849fd-2513-423f-9726-a21bdb67a8d0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:47.538896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:47.539614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.539899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.540179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.540886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:47.541899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:00:47.541899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 01 04:00:47.541899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: Jul 01 04:00:47.543104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:47.620395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 750/1498] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:00:47 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:00:47.624907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:47.636265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e27e5-2c8e-4a40-b9e5-c79f34125908 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:47.640957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:47.641621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.641903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.642180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.642964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:47.644096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:00:47.644096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 01 04:00:47.644096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: Jul 01 04:00:47.644978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:47.658683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 749/1499] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:00:47.663109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:47.673441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6012e08-375d-4937-9473-86a324616d06 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:47.678197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db882-8131-45a5-bf5f-db6dffb322d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:47.678856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db882-8131-45a5-bf5f-db6dffb322d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.679131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db882-8131-45a5-bf5f-db6dffb322d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.679439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db882-8131-45a5-bf5f-db6dffb322d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.680171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1db882-8131-45a5-bf5f-db6dffb322d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:47.681165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-da1db882-8131-45a5-bf5f-db6dffb322d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:00:47.681165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 01 04:00:47.681165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: Jul 01 04:00:47.682531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1db882-8131-45a5-bf5f-db6dffb322d5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:47.696938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 751/1500] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:00:47.702051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:47.713718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e489-a17e-4db1-babd-49137206ae0c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:47.718362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ed574-884f-4c91-9a90-b45f00a6e2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:47.719278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ed574-884f-4c91-9a90-b45f00a6e2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.719561 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ed574-884f-4c91-9a90-b45f00a6e2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.719835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ed574-884f-4c91-9a90-b45f00a6e2fa admin admin] PATH_INFO: `/v3/projects/50160f359da44938ba47e7353ee056db/users/c143eea0339d49eebeedc0ff43224cd8/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.736421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ed574-884f-4c91-9a90-b45f00a6e2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=50160f359da44938ba47e7353ee056db, user_id=c143eea0339d49eebeedc0ff43224cd8, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:47.737396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-fa2ed574-884f-4c91-9a90-b45f00a6e2fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:00:47.737396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 01 04:00:47.737396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: Jul 01 04:00:47.739410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ed574-884f-4c91-9a90-b45f00a6e2fa admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:47.799539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 750/1501] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:00:47 2022] PUT /identity/v3/projects/50160f359da44938ba47e7353ee056db/users/c143eea0339d49eebeedc0ff43224cd8/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:00:47.812697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f695524-2e40-423a-a161-e06061668309 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.812990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f695524-2e40-423a-a161-e06061668309 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.813267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f695524-2e40-423a-a161-e06061668309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.840775 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3f695524-2e40-423a-a161-e06061668309 None None] MFA Rules not processed for user `c143eea0339d49eebeedc0ff43224cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:00:47.842020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3f695524-2e40-423a-a161-e06061668309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:00:47.906753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 752/1502] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:00:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:00:47.920133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ee5cc-45e8-4ed8-8459-065291ca3020 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.920426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ee5cc-45e8-4ed8-8459-065291ca3020 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.920711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ee5cc-45e8-4ed8-8459-065291ca3020 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.922068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 751/1503] 10.222.0.20 () {54 vars in 963 bytes} [Fri Jul 1 04:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:00:47.926978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23da8d-bcba-44ef-bbb9-d8a50878b87e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:47.927293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23da8d-bcba-44ef-bbb9-d8a50878b87e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:47.927576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23da8d-bcba-44ef-bbb9-d8a50878b87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:47.946026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8a23da8d-bcba-44ef-bbb9-d8a50878b87e None None] MFA Rules not processed for user `ec8a755e84f04c9590c9c8d57781d2a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:00:47.947245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8a23da8d-bcba-44ef-bbb9-d8a50878b87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:00:47.997286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:00:48.004851 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 753/1504] 10.222.0.20 () {58 vars in 1060 bytes} [Fri Jul 1 04:00:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:00:48.012694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ee5cc-45e8-4ed8-8459-065291ca3020 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:48.024148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ee5cc-45e8-4ed8-8459-065291ca3020 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:48.028137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca88a4f-0e35-4cc9-aa17-8f4f095ebc90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:48.028886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca88a4f-0e35-4cc9-aa17-8f4f095ebc90 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:48.029209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca88a4f-0e35-4cc9-aa17-8f4f095ebc90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:48.029525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca88a4f-0e35-4cc9-aa17-8f4f095ebc90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:48.038800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca88a4f-0e35-4cc9-aa17-8f4f095ebc90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:48.040275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca88a4f-0e35-4cc9-aa17-8f4f095ebc90 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:48.050737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 752/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:00:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:00:48.094514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff88e-06d5-4da5-a1fc-2d6b152a3a10 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:48.094830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff88e-06d5-4da5-a1fc-2d6b152a3a10 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:48.095116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff88e-06d5-4da5-a1fc-2d6b152a3a10 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:48.096459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 754/1506] 10.222.0.20 () {52 vars in 829 bytes} [Fri Jul 1 04:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:00:48.101587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca88a4f-0e35-4cc9-aa17-8f4f095ebc90 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:48.164155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca88a4f-0e35-4cc9-aa17-8f4f095ebc90 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:48.169258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:48.170136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:48.170419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:48.170693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] PATH_INFO: `/v3/users/c143eea0339d49eebeedc0ff43224cd8` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:48.171908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c143eea0339d49eebeedc0ff43224cd8)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:48.172933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:00:48.172933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 01 04:00:48.172933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: Jul 01 04:00:48.174156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:48.426288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c143eea0339d49eebeedc0ff43224cd8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:48.426995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c143eea0339d49eebeedc0ff43224cd8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:48.427765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c143eea0339d49eebeedc0ff43224cd8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:48.446781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c143eea0339d49eebeedc0ff43224cd8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:48.459518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c143eea0339d49eebeedc0ff43224cd8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:48.464134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:00:48.650780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 753/1507] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:00:48 2022] DELETE /identity/v3/users/c143eea0339d49eebeedc0ff43224cd8 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:00:48.655848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff539bee-21b3-4eda-a5ae-17b4dadbce2d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:48.656426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff539bee-21b3-4eda-a5ae-17b4dadbce2d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:48.656519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff539bee-21b3-4eda-a5ae-17b4dadbce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:48.675579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ff539bee-21b3-4eda-a5ae-17b4dadbce2d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:00:48.676879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ff539bee-21b3-4eda-a5ae-17b4dadbce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:00:48.742800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 755/1508] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:00:48.752615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:48.754069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:00:48.817242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618f68b-07a0-4198-826b-a5bb1bf46055 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:48.820929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cca10a-dd18-4185-967c-49149c382739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:48.821578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca10a-dd18-4185-967c-49149c382739 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:48.821863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca10a-dd18-4185-967c-49149c382739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:48.822142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca10a-dd18-4185-967c-49149c382739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:48.832305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cca10a-dd18-4185-967c-49149c382739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:48.833762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cca10a-dd18-4185-967c-49149c382739 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:48.844793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 754/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:00:49.446034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff539bee-21b3-4eda-a5ae-17b4dadbce2d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:00:49.447576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ff539bee-21b3-4eda-a5ae-17b4dadbce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:00:49.459699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff539bee-21b3-4eda-a5ae-17b4dadbce2d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:00:49.464576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:00:49.465434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:00:49.465712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:00:49.465990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] PATH_INFO: `/v3/projects/50160f359da44938ba47e7353ee056db` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:00:49.467277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] RBAC: Authorizing `identity:delete_project(project_id=50160f359da44938ba47e7353ee056db)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:00:49.468251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:00:49.468251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 01 04:00:49.468251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: Jul 01 04:00:49.469448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:00:49.639809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50160f359da44938ba47e7353ee056db', 'request_id': 'req-d229abd4-1cfa-476d-b0a9-5f97f126923f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:49.645970 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50160f359da44938ba47e7353ee056db', 'request_id': 'req-d229abd4-1cfa-476d-b0a9-5f97f126923f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:49.659547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50160f359da44938ba47e7353ee056db', 'request_id': 'req-d229abd4-1cfa-476d-b0a9-5f97f126923f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:49.773457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] The token cache is being invalidate because project 50160f359da44938ba47e7353ee056db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:00:49.773747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:49.774400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d229abd4-1cfa-476d-b0a9-5f97f126923f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:00:49.776112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 756/1510] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:00:49 2022] DELETE /identity/v3/projects/50160f359da44938ba47e7353ee056db => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:00:58.839553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:00:59.770047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:02:27.809512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f98a4-89f8-417a-bf99-b4bb886d3cb4 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:27.810101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f98a4-89f8-417a-bf99-b4bb886d3cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:27.810101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f98a4-89f8-417a-bf99-b4bb886d3cb4 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:27.811853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 04:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:27.816856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b61b4-7911-4d8d-a0e0-a7b21b068987 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:27.817295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b61b4-7911-4d8d-a0e0-a7b21b068987 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:27.817439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b61b4-7911-4d8d-a0e0-a7b21b068987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:27.835109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e25b61b4-7911-4d8d-a0e0-a7b21b068987 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:27.836342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e25b61b4-7911-4d8d-a0e0-a7b21b068987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:27.896209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 04:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:02:28.051129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f98a4-89f8-417a-bf99-b4bb886d3cb4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:28.054984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-869f98a4-89f8-417a-bf99-b4bb886d3cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:28.128832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f98a4-89f8-417a-bf99-b4bb886d3cb4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:28.134248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af52ac6-1d00-4369-a95d-2b6194a18b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:28.134894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52ac6-1d00-4369-a95d-2b6194a18b06 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:28.135174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52ac6-1d00-4369-a95d-2b6194a18b06 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:28.135475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52ac6-1d00-4369-a95d-2b6194a18b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:28.143799 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af52ac6-1d00-4369-a95d-2b6194a18b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:28.145379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af52ac6-1d00-4369-a95d-2b6194a18b06 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:28.154811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 756/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:02:28.980708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe43ab-a1d4-42b0-b9c3-1ab8a24ac687 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:28.981247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe43ab-a1d4-42b0-b9c3-1ab8a24ac687 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:28.981247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe43ab-a1d4-42b0-b9c3-1ab8a24ac687 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:28.982566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 04:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:28.987737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895780b-451c-418b-942f-5b4e8ee77d81 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:28.988142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895780b-451c-418b-942f-5b4e8ee77d81 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:28.988295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895780b-451c-418b-942f-5b4e8ee77d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:29.008620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1895780b-451c-418b-942f-5b4e8ee77d81 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:29.009830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1895780b-451c-418b-942f-5b4e8ee77d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:29.020109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 04:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:02:29.173485 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe43ab-a1d4-42b0-b9c3-1ab8a24ac687 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:29.189838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe43ab-a1d4-42b0-b9c3-1ab8a24ac687 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:29.196580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0db3f-48c7-4b4c-ac42-0db7a9a1e16d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:29.196580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0db3f-48c7-4b4c-ac42-0db7a9a1e16d service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:29.196580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0db3f-48c7-4b4c-ac42-0db7a9a1e16d service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:29.196943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0db3f-48c7-4b4c-ac42-0db7a9a1e16d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:29.206095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0db3f-48c7-4b4c-ac42-0db7a9a1e16d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:29.207722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0db3f-48c7-4b4c-ac42-0db7a9a1e16d service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:29.217038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 759/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:02:30.129267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b43eb-fe1d-4f39-a2d3-0305d3dcd85d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:30.129845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b43eb-fe1d-4f39-a2d3-0305d3dcd85d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:30.129845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b43eb-fe1d-4f39-a2d3-0305d3dcd85d None None] PATH_INFO: `/v3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:30.131147 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 758/1517] 10.222.0.15 () {54 vars in 925 bytes} [Fri Jul 1 04:02:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:02:30.138771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78820f-ce32-4263-948a-adae8b14205a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:30.139161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78820f-ce32-4263-948a-adae8b14205a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:30.139380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78820f-ce32-4263-948a-adae8b14205a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:30.157795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6c78820f-ce32-4263-948a-adae8b14205a None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:30.159006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6c78820f-ce32-4263-948a-adae8b14205a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:30.222193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 760/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Jul 1 04:02:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:02:34.212877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d0395-2a2e-494d-a7d7-a1b53bd953dd None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:34.213479 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d0395-2a2e-494d-a7d7-a1b53bd953dd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:34.213479 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d0395-2a2e-494d-a7d7-a1b53bd953dd None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:34.214847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 759/1519] 10.222.0.20 () {54 vars in 960 bytes} [Fri Jul 1 04:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:34.220413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bfdd-a37e-4417-a56b-61315ff6db23 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:34.220862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bfdd-a37e-4417-a56b-61315ff6db23 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:34.220963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bfdd-a37e-4417-a56b-61315ff6db23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:34.239117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b157bfdd-a37e-4417-a56b-61315ff6db23 None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:34.239951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b157bfdd-a37e-4417-a56b-61315ff6db23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:34.248853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 761/1520] 10.222.0.20 () {58 vars in 1057 bytes} [Fri Jul 1 04:02:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:02:34.256371 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d0395-2a2e-494d-a7d7-a1b53bd953dd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:34.270022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d0395-2a2e-494d-a7d7-a1b53bd953dd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:34.274774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6181ac-3237-465e-a359-26894abe1e5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:34.275432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6181ac-3237-465e-a359-26894abe1e5d service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:34.275715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6181ac-3237-465e-a359-26894abe1e5d service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:34.276003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6181ac-3237-465e-a359-26894abe1e5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:34.286540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6181ac-3237-465e-a359-26894abe1e5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:34.288122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6181ac-3237-465e-a359-26894abe1e5d service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:34.299025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 760/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:02:40.435496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d9ee-eea9-4d2e-9bf3-6f81d172be3a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:40.436308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d9ee-eea9-4d2e-9bf3-6f81d172be3a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:40.436308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d9ee-eea9-4d2e-9bf3-6f81d172be3a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:40.438204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 04:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:40.444687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba102-fa78-4480-b9a5-bebc1f577694 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:40.445149 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba102-fa78-4480-b9a5-bebc1f577694 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:40.445253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba102-fa78-4480-b9a5-bebc1f577694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:40.463691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f5eba102-fa78-4480-b9a5-bebc1f577694 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:40.464915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f5eba102-fa78-4480-b9a5-bebc1f577694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:40.472818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 04:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:02:40.627568 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1d9ee-eea9-4d2e-9bf3-6f81d172be3a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:40.642141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1d9ee-eea9-4d2e-9bf3-6f81d172be3a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:40.647609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de594e-b87d-4849-8d9f-17e50a391a65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:40.648247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de594e-b87d-4849-8d9f-17e50a391a65 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:40.648532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de594e-b87d-4849-8d9f-17e50a391a65 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:40.648810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de594e-b87d-4849-8d9f-17e50a391a65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:40.657584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de594e-b87d-4849-8d9f-17e50a391a65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:40.662874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de594e-b87d-4849-8d9f-17e50a391a65 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:40.669747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 763/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:02:41.499980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa4367-8a2c-4f48-b7fc-5d88e325c4d4 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:41.500557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa4367-8a2c-4f48-b7fc-5d88e325c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:41.500557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa4367-8a2c-4f48-b7fc-5d88e325c4d4 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:41.501961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 04:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:41.506964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242d186-de5b-43f0-ae99-c0484c8c33be None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:41.507617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242d186-de5b-43f0-ae99-c0484c8c33be None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:41.507617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242d186-de5b-43f0-ae99-c0484c8c33be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:41.528049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e242d186-de5b-43f0-ae99-c0484c8c33be None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:41.529318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e242d186-de5b-43f0-ae99-c0484c8c33be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:41.538124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 04:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:02:41.690662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa4367-8a2c-4f48-b7fc-5d88e325c4d4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:41.706614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa4367-8a2c-4f48-b7fc-5d88e325c4d4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:41.712105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdef6be-2551-43b1-8bc1-0c1bc86b13d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:41.712757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdef6be-2551-43b1-8bc1-0c1bc86b13d3 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:41.713040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdef6be-2551-43b1-8bc1-0c1bc86b13d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:41.713363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdef6be-2551-43b1-8bc1-0c1bc86b13d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:41.722648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdef6be-2551-43b1-8bc1-0c1bc86b13d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:41.724248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdef6be-2551-43b1-8bc1-0c1bc86b13d3 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:41.733586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 763/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:02:45.661567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20671cd6-f55f-4b4a-8cac-6e4752c55f02 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:45.662164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20671cd6-f55f-4b4a-8cac-6e4752c55f02 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:45.662164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20671cd6-f55f-4b4a-8cac-6e4752c55f02 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:45.663534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 04:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:45.669258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9804190-4b73-4279-ba6e-13bbdf3e85b9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:45.669721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9804190-4b73-4279-ba6e-13bbdf3e85b9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:45.669821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9804190-4b73-4279-ba6e-13bbdf3e85b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:45.689008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f9804190-4b73-4279-ba6e-13bbdf3e85b9 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:45.690349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f9804190-4b73-4279-ba6e-13bbdf3e85b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:45.698817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 04:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:02:45.736631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20671cd6-f55f-4b4a-8cac-6e4752c55f02 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:45.738447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-20671cd6-f55f-4b4a-8cac-6e4752c55f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:45.796543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20671cd6-f55f-4b4a-8cac-6e4752c55f02 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:45.800170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c27fa-375c-4b39-ac60-b09642df8a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:45.800814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c27fa-375c-4b39-ac60-b09642df8a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:45.801096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c27fa-375c-4b39-ac60-b09642df8a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:45.801372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c27fa-375c-4b39-ac60-b09642df8a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:45.810394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c27fa-375c-4b39-ac60-b09642df8a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:45.812668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c27fa-375c-4b39-ac60-b09642df8a22 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:45.817696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:02:45.822507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 766/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:02:47.879400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4e4d0-327f-408e-b97a-954d999afb79 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:47.879963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4e4d0-327f-408e-b97a-954d999afb79 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:47.879963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4e4d0-327f-408e-b97a-954d999afb79 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:47.881340 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 04:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:47.886922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f807b-d917-40e7-a0eb-8332d6841c3d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:47.887422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f807b-d917-40e7-a0eb-8332d6841c3d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:47.887531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f807b-d917-40e7-a0eb-8332d6841c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:47.905364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e24f807b-d917-40e7-a0eb-8332d6841c3d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:47.907108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e24f807b-d917-40e7-a0eb-8332d6841c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:47.915961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 04:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:02:47.957241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4e4d0-327f-408e-b97a-954d999afb79 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:47.968769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4e4d0-327f-408e-b97a-954d999afb79 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:47.972182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf04d83-a363-4cad-8ebe-87af518db3d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:47.972811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf04d83-a363-4cad-8ebe-87af518db3d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:47.973087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf04d83-a363-4cad-8ebe-87af518db3d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:47.973355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf04d83-a363-4cad-8ebe-87af518db3d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:47.982409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf04d83-a363-4cad-8ebe-87af518db3d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:47.984411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf04d83-a363-4cad-8ebe-87af518db3d0 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:47.993722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 766/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:02:51.532648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6bdb-17be-48d2-96bd-511fbd44d550 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:51.533235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6bdb-17be-48d2-96bd-511fbd44d550 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:51.533235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6bdb-17be-48d2-96bd-511fbd44d550 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:51.534576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 04:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:51.539921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841eed8-061c-4056-94e8-beba7473992c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:51.540409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841eed8-061c-4056-94e8-beba7473992c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:51.540527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841eed8-061c-4056-94e8-beba7473992c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:51.546748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=62 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:02:51.559724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f841eed8-061c-4056-94e8-beba7473992c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:51.566488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f841eed8-061c-4056-94e8-beba7473992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:51.569544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 04:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:02:51.877961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a6bdb-17be-48d2-96bd-511fbd44d550 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:51.879673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dc0a6bdb-17be-48d2-96bd-511fbd44d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:51.953431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a6bdb-17be-48d2-96bd-511fbd44d550 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:51.956972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c60b75-6e23-4bde-a8a1-7c4d6e702464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:51.957614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c60b75-6e23-4bde-a8a1-7c4d6e702464 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:51.957893 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c60b75-6e23-4bde-a8a1-7c4d6e702464 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:51.958164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c60b75-6e23-4bde-a8a1-7c4d6e702464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:51.967115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c60b75-6e23-4bde-a8a1-7c4d6e702464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:51.968563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c60b75-6e23-4bde-a8a1-7c4d6e702464 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:51.977868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 769/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:02:54.067546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5e7eb-c402-4469-afb1-b099a6613f97 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:54.068126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5e7eb-c402-4469-afb1-b099a6613f97 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:54.068126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5e7eb-c402-4469-afb1-b099a6613f97 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:54.069546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 04:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:02:54.074740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e771-798b-4247-a70f-b9f6cebff301 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:54.075154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e771-798b-4247-a70f-b9f6cebff301 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:54.075338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e771-798b-4247-a70f-b9f6cebff301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:54.100915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-41d3e771-798b-4247-a70f-b9f6cebff301 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:02:54.102134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-41d3e771-798b-4247-a70f-b9f6cebff301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:54.110502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 04:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:02:54.414115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5e7eb-c402-4469-afb1-b099a6613f97 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:02:54.415939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eca5e7eb-c402-4469-afb1-b099a6613f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:02:54.429407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5e7eb-c402-4469-afb1-b099a6613f97 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:02:54.432969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe6670-fef3-445b-ab86-d9af4867c4a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:02:54.433633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe6670-fef3-445b-ab86-d9af4867c4a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:02:54.433910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe6670-fef3-445b-ab86-d9af4867c4a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:02:54.434182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe6670-fef3-445b-ab86-d9af4867c4a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:02:54.443947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe6670-fef3-445b-ab86-d9af4867c4a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:02:54.445397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe6670-fef3-445b-ab86-d9af4867c4a0 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:02:54.454900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 769/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:03:04.095417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=32 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:03:04.450689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=8 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:23.256473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01df2f7-f805-4156-b2a5-fee66c38a43a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.257640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01df2f7-f805-4156-b2a5-fee66c38a43a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.258144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01df2f7-f805-4156-b2a5-fee66c38a43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.282337 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d01df2f7-f805-4156-b2a5-fee66c38a43a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.284049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d01df2f7-f805-4156-b2a5-fee66c38a43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.294681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 771/1540] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:04:23.303809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f0b3-3572-4742-8328-116799c187ba None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.304392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f0b3-3572-4742-8328-116799c187ba None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.304392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f0b3-3572-4742-8328-116799c187ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.327419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1428f0b3-3572-4742-8328-116799c187ba None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.328654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1428f0b3-3572-4742-8328-116799c187ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.343648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 770/1541] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:04:23.348650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01df2f7-f805-4156-b2a5-fee66c38a43a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:23.363877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01df2f7-f805-4156-b2a5-fee66c38a43a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:23.375519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:23.376588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.377626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.378114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.379069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:23.379614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:23.381421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:23.399183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 772/1542] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:23.411000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428f0b3-3572-4742-8328-116799c187ba None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:23.421926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428f0b3-3572-4742-8328-116799c187ba None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:23.438155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3211c-b759-46ba-b611-a3ab0eb4425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:23.438884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3211c-b759-46ba-b611-a3ab0eb4425e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.439168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3211c-b759-46ba-b611-a3ab0eb4425e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.440036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3211c-b759-46ba-b611-a3ab0eb4425e admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.440780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3211c-b759-46ba-b611-a3ab0eb4425e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:23.442735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3211c-b759-46ba-b611-a3ab0eb4425e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:23.575364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 771/1543] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:23.579608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:23.597283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edb389-c6d0-4254-944c-94d066e8abc4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:23.599289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af506e-9893-4014-a3be-5fc536e734b5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.599289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af506e-9893-4014-a3be-5fc536e734b5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.599289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af506e-9893-4014-a3be-5fc536e734b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.622373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e9af506e-9893-4014-a3be-5fc536e734b5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.624362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e9af506e-9893-4014-a3be-5fc536e734b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.694995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd0a00-1b86-4fc7-8924-248ba71a1540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:23.697178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0a00-1b86-4fc7-8924-248ba71a1540 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.697776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0a00-1b86-4fc7-8924-248ba71a1540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.699472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0a00-1b86-4fc7-8924-248ba71a1540 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.700675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd0a00-1b86-4fc7-8924-248ba71a1540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:23.704397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd0a00-1b86-4fc7-8924-248ba71a1540 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:23.745710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 772/1544] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:23.747256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe9fdd-b4df-4358-a64b-9f13a0f73211 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.747687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe9fdd-b4df-4358-a64b-9f13a0f73211 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.747965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe9fdd-b4df-4358-a64b-9f13a0f73211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.769730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e2fe9fdd-b4df-4358-a64b-9f13a0f73211 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.776636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e2fe9fdd-b4df-4358-a64b-9f13a0f73211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.780307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 773/1545] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:23.781686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221fb19-1971-4032-b7b4-e92da7d3a265 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.781975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221fb19-1971-4032-b7b4-e92da7d3a265 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.782245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221fb19-1971-4032-b7b4-e92da7d3a265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.803462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6221fb19-1971-4032-b7b4-e92da7d3a265 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.805035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6221fb19-1971-4032-b7b4-e92da7d3a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.822845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 774/1546] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:23.824708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe406ec-df8b-4570-97da-7467dbf8f0b2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.825481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe406ec-df8b-4570-97da-7467dbf8f0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.826001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe406ec-df8b-4570-97da-7467dbf8f0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.838527 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 773/1547] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:23.846197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ab41-db85-4ff8-8574-88dcb6065324 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.846197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ab41-db85-4ff8-8574-88dcb6065324 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.846197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ab41-db85-4ff8-8574-88dcb6065324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.863584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7fe406ec-df8b-4570-97da-7467dbf8f0b2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.868160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7fe406ec-df8b-4570-97da-7467dbf8f0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.883400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7015ab41-db85-4ff8-8574-88dcb6065324 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.885469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7015ab41-db85-4ff8-8574-88dcb6065324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.885932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 775/1548] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:23.887106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a12b2-eccf-4eab-9271-a6965a0ebc7a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.887430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a12b2-eccf-4eab-9271-a6965a0ebc7a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.893723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a12b2-eccf-4eab-9271-a6965a0ebc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.896828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 774/1549] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:23.898800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806080-f8f1-43f6-a950-1474812a7415 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.899376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806080-f8f1-43f6-a950-1474812a7415 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.899933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806080-f8f1-43f6-a950-1474812a7415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.907921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-024a12b2-eccf-4eab-9271-a6965a0ebc7a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.909192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-024a12b2-eccf-4eab-9271-a6965a0ebc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.920976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 776/1550] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:23.920976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a12b2-eccf-4eab-9271-a6965a0ebc7a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:23.922373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-58806080-f8f1-43f6-a950-1474812a7415 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.924420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-58806080-f8f1-43f6-a950-1474812a7415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.935265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 775/1551] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:23.935480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb027e-b3e2-4fd4-874b-08a601e294f6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.935616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a12b2-eccf-4eab-9271-a6965a0ebc7a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:23.935841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb027e-b3e2-4fd4-874b-08a601e294f6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.936383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb027e-b3e2-4fd4-874b-08a601e294f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.944626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b0df1-e3f9-46d1-8b05-9c4d00136bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:23.945967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b0df1-e3f9-46d1-8b05-9c4d00136bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.946244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b0df1-e3f9-46d1-8b05-9c4d00136bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.947269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b0df1-e3f9-46d1-8b05-9c4d00136bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.948254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b0df1-e3f9-46d1-8b05-9c4d00136bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:23.951538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b0df1-e3f9-46d1-8b05-9c4d00136bfd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:23.961600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-54eb027e-b3e2-4fd4-874b-08a601e294f6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.963110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-54eb027e-b3e2-4fd4-874b-08a601e294f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.970140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 777/1552] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:23.973842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0e4c2-f5e8-4de0-926b-39023bb9a472 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.974019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 776/1553] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:23.974119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0e4c2-f5e8-4de0-926b-39023bb9a472 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.974119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0e4c2-f5e8-4de0-926b-39023bb9a472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.974286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9de273-cf19-42f5-a032-966ceb83f65c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:23.974286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9de273-cf19-42f5-a032-966ceb83f65c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:23.974286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9de273-cf19-42f5-a032-966ceb83f65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:23.993114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2a9de273-cf19-42f5-a032-966ceb83f65c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.997022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-68c0e4c2-f5e8-4de0-926b-39023bb9a472 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:23.997161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2a9de273-cf19-42f5-a032-966ceb83f65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:23.997269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-68c0e4c2-f5e8-4de0-926b-39023bb9a472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:24.002557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 777/1554] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:24.002971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 778/1555] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:24.003252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9de273-cf19-42f5-a032-966ceb83f65c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.003831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0e4c2-f5e8-4de0-926b-39023bb9a472 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.014039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0e4c2-f5e8-4de0-926b-39023bb9a472 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.019349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9de273-cf19-42f5-a032-966ceb83f65c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.019556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.020046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.020321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.020591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.021327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:24.021636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.022648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:04:24.022648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 01 04:04:24.022648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: Jul 01 04:04:24.023556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.024371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f18ee8-5ff9-4bc3-8532-170aa72a653c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.025020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f18ee8-5ff9-4bc3-8532-170aa72a653c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.025436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f18ee8-5ff9-4bc3-8532-170aa72a653c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.025726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f18ee8-5ff9-4bc3-8532-170aa72a653c admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.026457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f18ee8-5ff9-4bc3-8532-170aa72a653c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:24.026883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f18ee8-5ff9-4bc3-8532-170aa72a653c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.028542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f18ee8-5ff9-4bc3-8532-170aa72a653c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.036581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 779/1556] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:24.036944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.042200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 778/1557] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:24.044315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfabd6e-1b95-400e-9167-8e4f95c5663c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.044848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfabd6e-1b95-400e-9167-8e4f95c5663c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.045057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfabd6e-1b95-400e-9167-8e4f95c5663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.046903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c37345-ad71-40b7-ac15-8913764532d6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.052408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.053074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.053354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.053634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.054384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:24.054687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.057162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.065302 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5bfabd6e-1b95-400e-9167-8e4f95c5663c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:24.066536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5bfabd6e-1b95-400e-9167-8e4f95c5663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:24.070803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 780/1558] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:24.072039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.081390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 779/1559] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:24.082400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655ebbd-6b31-4d90-9d78-3d1f0d193367 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.083001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6692814-c749-403b-bb1e-e2368b19053a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.083381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6692814-c749-403b-bb1e-e2368b19053a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.083713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6692814-c749-403b-bb1e-e2368b19053a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.087625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5648ca-03f9-4cd7-ac70-e71fe41ab3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.088399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5648ca-03f9-4cd7-ac70-e71fe41ab3ad admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.088825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5648ca-03f9-4cd7-ac70-e71fe41ab3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.089111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5648ca-03f9-4cd7-ac70-e71fe41ab3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.089835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5648ca-03f9-4cd7-ac70-e71fe41ab3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.092571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5648ca-03f9-4cd7-ac70-e71fe41ab3ad admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.103420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 781/1560] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:24.104246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5648ca-03f9-4cd7-ac70-e71fe41ab3ad admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.114666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5648ca-03f9-4cd7-ac70-e71fe41ab3ad admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.119485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce787b5e-0c61-4b59-ace3-f0c4c045a4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.120138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce787b5e-0c61-4b59-ace3-f0c4c045a4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.120418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce787b5e-0c61-4b59-ace3-f0c4c045a4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.120690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce787b5e-0c61-4b59-ace3-f0c4c045a4ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.121405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce787b5e-0c61-4b59-ace3-f0c4c045a4ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:24.121706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce787b5e-0c61-4b59-ace3-f0c4c045a4ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.123349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce787b5e-0c61-4b59-ace3-f0c4c045a4ed admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.126437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d6692814-c749-403b-bb1e-e2368b19053a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:24.127692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d6692814-c749-403b-bb1e-e2368b19053a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:24.135794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 780/1562] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:24.135959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 782/1562] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:24.139583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c881e4f-4233-4565-aaee-f0d78c389e3d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.139583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c881e4f-4233-4565-aaee-f0d78c389e3d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.139583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c881e4f-4233-4565-aaee-f0d78c389e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.140907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6692814-c749-403b-bb1e-e2368b19053a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.150635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6692814-c749-403b-bb1e-e2368b19053a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.155191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.155869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.155998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6c881e4f-4233-4565-aaee-f0d78c389e3d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:24.156142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.156412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.157126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.157241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6c881e4f-4233-4565-aaee-f0d78c389e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:24.158625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:04:24.158625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 01 04:04:24.158625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: Jul 01 04:04:24.159847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.165983 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 783/1563] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:24.166777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c881e4f-4233-4565-aaee-f0d78c389e3d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.180783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c881e4f-4233-4565-aaee-f0d78c389e3d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.185530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9d45e-4a8c-4bfd-9dd8-df3526ed254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.186442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9d45e-4a8c-4bfd-9dd8-df3526ed254b admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.186722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9d45e-4a8c-4bfd-9dd8-df3526ed254b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.186990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9d45e-4a8c-4bfd-9dd8-df3526ed254b admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.187745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9d45e-4a8c-4bfd-9dd8-df3526ed254b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.190223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9d45e-4a8c-4bfd-9dd8-df3526ed254b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.233101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 781/1564] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:24.233883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.274399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 784/1565] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:24.276435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788473d-297d-4360-a7f0-87c0181a1103 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.276728 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788473d-297d-4360-a7f0-87c0181a1103 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.277429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788473d-297d-4360-a7f0-87c0181a1103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.291745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:24.296264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b788473d-297d-4360-a7f0-87c0181a1103 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:24.296532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819a7d7-010e-453a-aa65-ea0268db5af5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.297478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b788473d-297d-4360-a7f0-87c0181a1103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:24.301912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:24.354343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842c28d-2013-4496-83bb-0a22bc997b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.354996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842c28d-2013-4496-83bb-0a22bc997b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.355338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842c28d-2013-4496-83bb-0a22bc997b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.355619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842c28d-2013-4496-83bb-0a22bc997b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.357793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842c28d-2013-4496-83bb-0a22bc997b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.357932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 785/1566] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:24.357932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788473d-297d-4360-a7f0-87c0181a1103 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.358558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842c28d-2013-4496-83bb-0a22bc997b5b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.369726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788473d-297d-4360-a7f0-87c0181a1103 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.374571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954514c8-8659-4c76-be4c-db53fbf3bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.375502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954514c8-8659-4c76-be4c-db53fbf3bb62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.375785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954514c8-8659-4c76-be4c-db53fbf3bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.376059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954514c8-8659-4c76-be4c-db53fbf3bb62 admin admin] PATH_INFO: `/v3/projects/cbc23d0f41844c50af19f78ee6bf409d/users/2055155c6a7f49709124b2f00281c14f/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.391704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954514c8-8659-4c76-be4c-db53fbf3bb62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc23d0f41844c50af19f78ee6bf409d, user_id=2055155c6a7f49709124b2f00281c14f, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.394632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954514c8-8659-4c76-be4c-db53fbf3bb62 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.520168 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 782/1567] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/projects => generated 389 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:24.521016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842c28d-2013-4496-83bb-0a22bc997b5b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.573126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 786/1568] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:24 2022] PUT /identity/v3/projects/cbc23d0f41844c50af19f78ee6bf409d/users/2055155c6a7f49709124b2f00281c14f/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:24.573968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954514c8-8659-4c76-be4c-db53fbf3bb62 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.597275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842c28d-2013-4496-83bb-0a22bc997b5b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.641229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954514c8-8659-4c76-be4c-db53fbf3bb62 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.646734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aefdee-98e1-4e25-8669-c91c5ece7559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.647419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aefdee-98e1-4e25-8669-c91c5ece7559 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.647700 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aefdee-98e1-4e25-8669-c91c5ece7559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.647969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aefdee-98e1-4e25-8669-c91c5ece7559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.648721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aefdee-98e1-4e25-8669-c91c5ece7559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.650610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aefdee-98e1-4e25-8669-c91c5ece7559 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.655200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.655908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.656192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.656468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.657195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:24.657506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.659696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.671422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 783/1569] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:24.672285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.682327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388b1c4-4cd0-4264-8213-8c141bd83b73 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.687250 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.687904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.688184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.688457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.689196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:24.689557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.691161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.704113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 784/1570] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:24.705533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.715831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef930aed-5018-47ad-9280-6bca57954ffb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.721490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42371d-7d56-41c2-88b1-c3ef1c2a34e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.722248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42371d-7d56-41c2-88b1-c3ef1c2a34e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.722529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42371d-7d56-41c2-88b1-c3ef1c2a34e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.722803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42371d-7d56-41c2-88b1-c3ef1c2a34e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.723519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42371d-7d56-41c2-88b1-c3ef1c2a34e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.725412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42371d-7d56-41c2-88b1-c3ef1c2a34e0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.747695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 787/1571] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:24.747695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aefdee-98e1-4e25-8669-c91c5ece7559 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.807451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aefdee-98e1-4e25-8669-c91c5ece7559 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.812491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d1d75-7349-4f1c-9463-46e813d7936c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.813176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d1d75-7349-4f1c-9463-46e813d7936c admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.813493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d1d75-7349-4f1c-9463-46e813d7936c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.813773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d1d75-7349-4f1c-9463-46e813d7936c admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.814989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d1d75-7349-4f1c-9463-46e813d7936c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.816060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-541d1d75-7349-4f1c-9463-46e813d7936c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:04:24.816060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 01 04:04:24.816060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: Jul 01 04:04:24.817257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d1d75-7349-4f1c-9463-46e813d7936c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.821043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:24.827666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 785/1572] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:24.828620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42371d-7d56-41c2-88b1-c3ef1c2a34e0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.838928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42371d-7d56-41c2-88b1-c3ef1c2a34e0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.843431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.844077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.844355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.844629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.845340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:24.845645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.847214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.858885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 786/1573] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:24.859853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.871287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26967f0-ba0b-4aa2-a868-b614606a62e4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.876173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c059d7-49c3-4080-af11-109b1fce375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.877393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c059d7-49c3-4080-af11-109b1fce375d admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.877667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c059d7-49c3-4080-af11-109b1fce375d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.877938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c059d7-49c3-4080-af11-109b1fce375d admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.878629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c059d7-49c3-4080-af11-109b1fce375d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.880590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c059d7-49c3-4080-af11-109b1fce375d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:24.946788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:24.952109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 788/1574] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:24.956982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24a49b-6e43-470b-82ae-12fb4b100233 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.956982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24a49b-6e43-470b-82ae-12fb4b100233 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.956982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24a49b-6e43-470b-82ae-12fb4b100233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.978624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 787/1575] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:24.978624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c059d7-49c3-4080-af11-109b1fce375d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:24.980183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7f24a49b-6e43-470b-82ae-12fb4b100233 None None] MFA Rules not processed for user `2055155c6a7f49709124b2f00281c14f`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:24.981393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7f24a49b-6e43-470b-82ae-12fb4b100233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:24.988352 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c059d7-49c3-4080-af11-109b1fce375d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:24.993549 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e994c6-cddb-4828-a24b-b9287b1abac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:24.994202 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e994c6-cddb-4828-a24b-b9287b1abac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:24.994487 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e994c6-cddb-4828-a24b-b9287b1abac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:24.994760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e994c6-cddb-4828-a24b-b9287b1abac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:24.995516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e994c6-cddb-4828-a24b-b9287b1abac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:24.997923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e994c6-cddb-4828-a24b-b9287b1abac5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.050472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 789/1576] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:25.051386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24a49b-6e43-470b-82ae-12fb4b100233 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.065912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24a49b-6e43-470b-82ae-12fb4b100233 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.070198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcb162-32a6-4656-837a-4f5dac46044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.071085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcb162-32a6-4656-837a-4f5dac46044a admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.071522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcb162-32a6-4656-837a-4f5dac46044a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.071896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcb162-32a6-4656-837a-4f5dac46044a admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.072894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcb162-32a6-4656-837a-4f5dac46044a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.075620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcb162-32a6-4656-837a-4f5dac46044a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.122182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 788/1577] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:24 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:25.123030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e994c6-cddb-4828-a24b-b9287b1abac5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.188888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e994c6-cddb-4828-a24b-b9287b1abac5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.194106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dec3d4-977d-49f5-873f-abeea45fc2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.194772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec3d4-977d-49f5-873f-abeea45fc2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.195052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec3d4-977d-49f5-873f-abeea45fc2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.195351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec3d4-977d-49f5-873f-abeea45fc2f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.196038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dec3d4-977d-49f5-873f-abeea45fc2f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.197941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dec3d4-977d-49f5-873f-abeea45fc2f7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.214344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 790/1578] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:25 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:25.215106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcb162-32a6-4656-837a-4f5dac46044a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.284483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcb162-32a6-4656-837a-4f5dac46044a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.289411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1a563-a583-40a6-bd44-780e38f71be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.290090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a563-a583-40a6-bd44-780e38f71be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.290379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a563-a583-40a6-bd44-780e38f71be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.290652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a563-a583-40a6-bd44-780e38f71be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.291453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1a563-a583-40a6-bd44-780e38f71be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.294772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1a563-a583-40a6-bd44-780e38f71be7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.294913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 789/1579] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:25 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:25.294913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dec3d4-977d-49f5-873f-abeea45fc2f7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.304282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dec3d4-977d-49f5-873f-abeea45fc2f7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.308195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 791/1580] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:25.308195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1a563-a583-40a6-bd44-780e38f71be7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.312421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239622f7-cde2-439a-afc6-c5a3aaab9871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.313099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239622f7-cde2-439a-afc6-c5a3aaab9871 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.313403 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239622f7-cde2-439a-afc6-c5a3aaab9871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.313689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239622f7-cde2-439a-afc6-c5a3aaab9871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.314961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239622f7-cde2-439a-afc6-c5a3aaab9871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.316873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1a563-a583-40a6-bd44-780e38f71be7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.317094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239622f7-cde2-439a-afc6-c5a3aaab9871 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.321647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f9577-e5fc-41d7-80ee-b0dc32031eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.322315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9577-e5fc-41d7-80ee-b0dc32031eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.322600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9577-e5fc-41d7-80ee-b0dc32031eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.322872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9577-e5fc-41d7-80ee-b0dc32031eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.323636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f9577-e5fc-41d7-80ee-b0dc32031eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.325224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f9577-e5fc-41d7-80ee-b0dc32031eaa admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.336124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 792/1581] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:25.336977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f9577-e5fc-41d7-80ee-b0dc32031eaa admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.348434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f9577-e5fc-41d7-80ee-b0dc32031eaa admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.353300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d56385-44b9-4e46-9f88-57cbd88cff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.353964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d56385-44b9-4e46-9f88-57cbd88cff04 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.354245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d56385-44b9-4e46-9f88-57cbd88cff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.354518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d56385-44b9-4e46-9f88-57cbd88cff04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.355262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d56385-44b9-4e46-9f88-57cbd88cff04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.356862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d56385-44b9-4e46-9f88-57cbd88cff04 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.367724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 793/1582] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:25.368527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d56385-44b9-4e46-9f88-57cbd88cff04 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.379164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d56385-44b9-4e46-9f88-57cbd88cff04 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.383827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd41e7-d538-4bb1-8c85-9450b8ad0c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.384479 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd41e7-d538-4bb1-8c85-9450b8ad0c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.384757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd41e7-d538-4bb1-8c85-9450b8ad0c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.385029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd41e7-d538-4bb1-8c85-9450b8ad0c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.385719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd41e7-d538-4bb1-8c85-9450b8ad0c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.387639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd41e7-d538-4bb1-8c85-9450b8ad0c61 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.437192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 790/1583] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:25 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:25.438377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239622f7-cde2-439a-afc6-c5a3aaab9871 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.512068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239622f7-cde2-439a-afc6-c5a3aaab9871 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.517011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d217b7-3a0b-4dcc-80f9-3471e03e92c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.518298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d217b7-3a0b-4dcc-80f9-3471e03e92c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.518580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d217b7-3a0b-4dcc-80f9-3471e03e92c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.518895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d217b7-3a0b-4dcc-80f9-3471e03e92c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.519687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d217b7-3a0b-4dcc-80f9-3471e03e92c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.521532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d217b7-3a0b-4dcc-80f9-3471e03e92c0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.648311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 794/1584] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:25 2022] POST /identity/v3/projects => generated 387 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:25.649312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd41e7-d538-4bb1-8c85-9450b8ad0c61 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.656109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 791/1585] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:25 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:25.657229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d217b7-3a0b-4dcc-80f9-3471e03e92c0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.663153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:25.668711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d217b7-3a0b-4dcc-80f9-3471e03e92c0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.677371 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:25.719710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd41e7-d538-4bb1-8c85-9450b8ad0c61 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.725797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062f8ae-aa14-4e3d-97e3-762e289fcc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.726509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062f8ae-aa14-4e3d-97e3-762e289fcc03 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.726780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062f8ae-aa14-4e3d-97e3-762e289fcc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.727050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062f8ae-aa14-4e3d-97e3-762e289fcc03 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.727762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062f8ae-aa14-4e3d-97e3-762e289fcc03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.729779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062f8ae-aa14-4e3d-97e3-762e289fcc03 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.735375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5bff3-a301-46f9-af1a-f75255b60edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.736137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5bff3-a301-46f9-af1a-f75255b60edc admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.736935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5bff3-a301-46f9-af1a-f75255b60edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.737213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5bff3-a301-46f9-af1a-f75255b60edc admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.737947 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5bff3-a301-46f9-af1a-f75255b60edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.739588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5bff3-a301-46f9-af1a-f75255b60edc admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.750971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 792/1586] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:25.756980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5bff3-a301-46f9-af1a-f75255b60edc admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.762001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5bff3-a301-46f9-af1a-f75255b60edc admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.766648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187b332-b4c7-4995-aecb-60a1639ed3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.767375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187b332-b4c7-4995-aecb-60a1639ed3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.767665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187b332-b4c7-4995-aecb-60a1639ed3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.767943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187b332-b4c7-4995-aecb-60a1639ed3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.768740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187b332-b4c7-4995-aecb-60a1639ed3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.770324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187b332-b4c7-4995-aecb-60a1639ed3c6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.780484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 793/1587] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:25.781517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187b332-b4c7-4995-aecb-60a1639ed3c6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.795033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187b332-b4c7-4995-aecb-60a1639ed3c6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.800160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6749b-4ed8-4ef5-9f63-49f870546015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.800811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6749b-4ed8-4ef5-9f63-49f870546015 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.801095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6749b-4ed8-4ef5-9f63-49f870546015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.801366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6749b-4ed8-4ef5-9f63-49f870546015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.802158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d6749b-4ed8-4ef5-9f63-49f870546015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.803719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d6749b-4ed8-4ef5-9f63-49f870546015 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.814561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 794/1588] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:25.815393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6749b-4ed8-4ef5-9f63-49f870546015 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.825112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6749b-4ed8-4ef5-9f63-49f870546015 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.829752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb057ea7-22cb-4f15-b774-8349fc65c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.830885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb057ea7-22cb-4f15-b774-8349fc65c4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.831161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb057ea7-22cb-4f15-b774-8349fc65c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.831460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb057ea7-22cb-4f15-b774-8349fc65c4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.832176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb057ea7-22cb-4f15-b774-8349fc65c4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.833851 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb057ea7-22cb-4f15-b774-8349fc65c4e7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.843743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 795/1589] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:25.844942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb057ea7-22cb-4f15-b774-8349fc65c4e7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.853360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 795/1590] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:25 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:25.854252 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062f8ae-aa14-4e3d-97e3-762e289fcc03 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.855089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb057ea7-22cb-4f15-b774-8349fc65c4e7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.860045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748566e-7dbb-4adb-9daf-a20630949bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.860752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748566e-7dbb-4adb-9daf-a20630949bae admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.861026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748566e-7dbb-4adb-9daf-a20630949bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.861936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748566e-7dbb-4adb-9daf-a20630949bae admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.862098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748566e-7dbb-4adb-9daf-a20630949bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.864041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748566e-7dbb-4adb-9daf-a20630949bae admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.865281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062f8ae-aa14-4e3d-97e3-762e289fcc03 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.869954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52702d-edf1-45ce-b38f-ffc90a79559b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:25.870771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52702d-edf1-45ce-b38f-ffc90a79559b admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:25.871053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52702d-edf1-45ce-b38f-ffc90a79559b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:25.871361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52702d-edf1-45ce-b38f-ffc90a79559b admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:25.872058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52702d-edf1-45ce-b38f-ffc90a79559b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:25.874033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52702d-edf1-45ce-b38f-ffc90a79559b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:25.983753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 796/1591] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:25 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:25.984502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748566e-7dbb-4adb-9daf-a20630949bae admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:25.994612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748566e-7dbb-4adb-9daf-a20630949bae admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:25.999375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee999f2-cb76-4a62-b2d4-b63875576499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.000031 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee999f2-cb76-4a62-b2d4-b63875576499 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.000309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee999f2-cb76-4a62-b2d4-b63875576499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.000579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee999f2-cb76-4a62-b2d4-b63875576499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.001294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee999f2-cb76-4a62-b2d4-b63875576499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.002926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee999f2-cb76-4a62-b2d4-b63875576499 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.012370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 796/1592] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:25 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:26.020144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52702d-edf1-45ce-b38f-ffc90a79559b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.020293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 797/1593] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:26.020293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee999f2-cb76-4a62-b2d4-b63875576499 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.025686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee999f2-cb76-4a62-b2d4-b63875576499 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.025841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52702d-edf1-45ce-b38f-ffc90a79559b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.030519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7956b9-7a9d-4de6-8663-f97d46298374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.030663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.031190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7956b9-7a9d-4de6-8663-f97d46298374 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.031508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7956b9-7a9d-4de6-8663-f97d46298374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.031605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.031784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7956b9-7a9d-4de6-8663-f97d46298374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.031878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.032059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] PATH_INFO: `/v3/projects/cb56c390ed7649479455d6dc3904deb0/users/8ab2da92b7e94592b80ef3d7852f51ae/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.032705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7956b9-7a9d-4de6-8663-f97d46298374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.035004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7956b9-7a9d-4de6-8663-f97d46298374 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.045803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 797/1594] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:26.046723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7956b9-7a9d-4de6-8663-f97d46298374 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.053889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb56c390ed7649479455d6dc3904deb0, user_id=8ab2da92b7e94592b80ef3d7852f51ae, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.054935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:04:26.054935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 01 04:04:26.054935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: Jul 01 04:04:26.056619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7956b9-7a9d-4de6-8663-f97d46298374 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.056910 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.061559 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42948-d966-4f6b-8220-4dd345ddbdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.062435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42948-d966-4f6b-8220-4dd345ddbdc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.062719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42948-d966-4f6b-8220-4dd345ddbdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.062995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42948-d966-4f6b-8220-4dd345ddbdc4 admin admin] PATH_INFO: `/v3/projects/f0143140d4f649478e36abcee284640c/users/8622ec5212c84d87b9b7c8f725012a07/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.079397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a42948-d966-4f6b-8220-4dd345ddbdc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0143140d4f649478e36abcee284640c, user_id=8622ec5212c84d87b9b7c8f725012a07, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.082617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a42948-d966-4f6b-8220-4dd345ddbdc4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.163779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 798/1595] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:26 2022] PUT /identity/v3/projects/cb56c390ed7649479455d6dc3904deb0/users/8ab2da92b7e94592b80ef3d7852f51ae/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:26.164701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.186390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 798/1596] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:26 2022] PUT /identity/v3/projects/f0143140d4f649478e36abcee284640c/users/8622ec5212c84d87b9b7c8f725012a07/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:26.187469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42948-d966-4f6b-8220-4dd345ddbdc4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.230054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29ce9d-9bcf-49a8-a10a-48efd8243e89 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.247390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:26.255470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42948-d966-4f6b-8220-4dd345ddbdc4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.260277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288889ff-7dfd-490c-922a-960dfebe0b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.263645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288889ff-7dfd-490c-922a-960dfebe0b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.263645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288889ff-7dfd-490c-922a-960dfebe0b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.263645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288889ff-7dfd-490c-922a-960dfebe0b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.263645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288889ff-7dfd-490c-922a-960dfebe0b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.264013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288889ff-7dfd-490c-922a-960dfebe0b3d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.275906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 799/1597] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:26.277043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288889ff-7dfd-490c-922a-960dfebe0b3d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.287344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288889ff-7dfd-490c-922a-960dfebe0b3d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.292022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcf680-32ca-4a7f-a721-cb6d23643ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.292466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2538dee-6c65-42c8-aea0-00a32bc538ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.292816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcf680-32ca-4a7f-a721-cb6d23643ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.293097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcf680-32ca-4a7f-a721-cb6d23643ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.293191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2538dee-6c65-42c8-aea0-00a32bc538ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.293370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcf680-32ca-4a7f-a721-cb6d23643ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.293465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2538dee-6c65-42c8-aea0-00a32bc538ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.293650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2538dee-6c65-42c8-aea0-00a32bc538ca admin admin] PATH_INFO: `/v3/projects/9dccdb0188264f2686d7275e19f39d05/users/708734ba3ddc49a3a9dba1d891aeccc7/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.294096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcf680-32ca-4a7f-a721-cb6d23643ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.295745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcf680-32ca-4a7f-a721-cb6d23643ae9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.306355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 800/1598] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:26.307199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcf680-32ca-4a7f-a721-cb6d23643ae9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.308928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2538dee-6c65-42c8-aea0-00a32bc538ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dccdb0188264f2686d7275e19f39d05, user_id=708734ba3ddc49a3a9dba1d891aeccc7, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.311943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2538dee-6c65-42c8-aea0-00a32bc538ca admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.318577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcf680-32ca-4a7f-a721-cb6d23643ae9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.323354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df93a8d-2dc1-480b-8c80-86259ad52dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.324015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93a8d-2dc1-480b-8c80-86259ad52dde admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.331845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93a8d-2dc1-480b-8c80-86259ad52dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.331845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93a8d-2dc1-480b-8c80-86259ad52dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.331845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df93a8d-2dc1-480b-8c80-86259ad52dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.331845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df93a8d-2dc1-480b-8c80-86259ad52dde admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.337945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 801/1599] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:26.338748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df93a8d-2dc1-480b-8c80-86259ad52dde admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.348172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df93a8d-2dc1-480b-8c80-86259ad52dde admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.353239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817a411-c42f-45de-9025-1554f8ab6b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.353897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817a411-c42f-45de-9025-1554f8ab6b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.354174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817a411-c42f-45de-9025-1554f8ab6b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.354445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817a411-c42f-45de-9025-1554f8ab6b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.355166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817a411-c42f-45de-9025-1554f8ab6b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.356831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817a411-c42f-45de-9025-1554f8ab6b96 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.366986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 802/1600] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:26.367804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817a411-c42f-45de-9025-1554f8ab6b96 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.378882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817a411-c42f-45de-9025-1554f8ab6b96 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.386078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 799/1601] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:26 2022] PUT /identity/v3/projects/9dccdb0188264f2686d7275e19f39d05/users/708734ba3ddc49a3a9dba1d891aeccc7/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:26.389259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60fc58-ab8b-47a0-8a03-3f1ca4f35892 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.389259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60fc58-ab8b-47a0-8a03-3f1ca4f35892 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.389462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60fc58-ab8b-47a0-8a03-3f1ca4f35892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.405696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:26.408958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-da60fc58-ab8b-47a0-8a03-3f1ca4f35892 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:26.410178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-da60fc58-ab8b-47a0-8a03-3f1ca4f35892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:26.446087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7aafe-7175-44cb-91c9-fa85cb856c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.447041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7aafe-7175-44cb-91c9-fa85cb856c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.447352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7aafe-7175-44cb-91c9-fa85cb856c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.447628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7aafe-7175-44cb-91c9-fa85cb856c4a admin admin] PATH_INFO: `/v3/projects/74f993901b2b497d904ad1e4343954da/users/b191071f184241908b1260b46c35a8ce/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.462978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7aafe-7175-44cb-91c9-fa85cb856c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f993901b2b497d904ad1e4343954da, user_id=b191071f184241908b1260b46c35a8ce, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.465879 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7aafe-7175-44cb-91c9-fa85cb856c4a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.471821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 800/1602] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:26.473279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52063142-0dbf-4ba3-a420-365e257f5413 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.473569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52063142-0dbf-4ba3-a420-365e257f5413 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.473852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52063142-0dbf-4ba3-a420-365e257f5413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.492328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-52063142-0dbf-4ba3-a420-365e257f5413 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:26.493541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-52063142-0dbf-4ba3-a420-365e257f5413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:26.501236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 801/1603] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:26.502151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52063142-0dbf-4ba3-a420-365e257f5413 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.512332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52063142-0dbf-4ba3-a420-365e257f5413 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.517186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11385c37-85db-48c0-b4b7-cc881027ee1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.517908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385c37-85db-48c0-b4b7-cc881027ee1b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.518196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385c37-85db-48c0-b4b7-cc881027ee1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.518477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385c37-85db-48c0-b4b7-cc881027ee1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.519270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11385c37-85db-48c0-b4b7-cc881027ee1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.520937 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11385c37-85db-48c0-b4b7-cc881027ee1b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.531613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 802/1604] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:26.533423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11385c37-85db-48c0-b4b7-cc881027ee1b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.544008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11385c37-85db-48c0-b4b7-cc881027ee1b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.549167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4523c-e50c-4c47-a2b5-7a14bf0e049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.549904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4523c-e50c-4c47-a2b5-7a14bf0e049f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.550198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4523c-e50c-4c47-a2b5-7a14bf0e049f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.550475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4523c-e50c-4c47-a2b5-7a14bf0e049f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.551207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4523c-e50c-4c47-a2b5-7a14bf0e049f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.552853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4523c-e50c-4c47-a2b5-7a14bf0e049f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.558306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 803/1605] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:26 2022] PUT /identity/v3/projects/74f993901b2b497d904ad1e4343954da/users/b191071f184241908b1260b46c35a8ce/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:26.559166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7aafe-7175-44cb-91c9-fa85cb856c4a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.563935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 803/1606] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:26.564799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4523c-e50c-4c47-a2b5-7a14bf0e049f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.575669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4523c-e50c-4c47-a2b5-7a14bf0e049f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.633847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7aafe-7175-44cb-91c9-fa85cb856c4a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.634105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a3fe5-c256-4b0c-90f6-ba66fab78aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.634822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3fe5-c256-4b0c-90f6-ba66fab78aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.635108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3fe5-c256-4b0c-90f6-ba66fab78aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.635419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3fe5-c256-4b0c-90f6-ba66fab78aa0 admin admin] PATH_INFO: `/v3/projects/f80f7c3079364d238d394945f393bb75/users/57dd68408eb14eda9586e4d1283a563e/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.638618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f3c5d-468a-4638-be67-1f13b9186452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.639569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f3c5d-468a-4638-be67-1f13b9186452 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.639852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f3c5d-468a-4638-be67-1f13b9186452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.640124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f3c5d-468a-4638-be67-1f13b9186452 admin admin] PATH_INFO: `/v3/projects/3aabfc8294b342dab1026cf8cd8b570a/users/93641feb18044278b072f9288d47c46b/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.651459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a3fe5-c256-4b0c-90f6-ba66fab78aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80f7c3079364d238d394945f393bb75, user_id=57dd68408eb14eda9586e4d1283a563e, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.654614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a3fe5-c256-4b0c-90f6-ba66fab78aa0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.656582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f3c5d-468a-4638-be67-1f13b9186452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aabfc8294b342dab1026cf8cd8b570a, user_id=93641feb18044278b072f9288d47c46b, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.659917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f3c5d-468a-4638-be67-1f13b9186452 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.735255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:26.741016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 804/1607] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:26 2022] PUT /identity/v3/projects/f80f7c3079364d238d394945f393bb75/users/57dd68408eb14eda9586e4d1283a563e/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:26.742466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cc0d-5041-467f-989b-46d66d53ced6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.742893 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cc0d-5041-467f-989b-46d66d53ced6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.743133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cc0d-5041-467f-989b-46d66d53ced6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.756015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 804/1608] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:26 2022] PUT /identity/v3/projects/3aabfc8294b342dab1026cf8cd8b570a/users/93641feb18044278b072f9288d47c46b/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:26.756015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f3c5d-468a-4638-be67-1f13b9186452 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.761739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-97e2cc0d-5041-467f-989b-46d66d53ced6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:26.762980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-97e2cc0d-5041-467f-989b-46d66d53ced6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:26.817191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f3c5d-468a-4638-be67-1f13b9186452 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.821027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ef5e-1e4b-4384-90a1-7424923ee2e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.821764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ef5e-1e4b-4384-90a1-7424923ee2e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.822080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ef5e-1e4b-4384-90a1-7424923ee2e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.822389 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ef5e-1e4b-4384-90a1-7424923ee2e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.829463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 805/1609] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:26.831708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cc0d-5041-467f-989b-46d66d53ced6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.832892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87ef5e-1e4b-4384-90a1-7424923ee2e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.834340 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87ef5e-1e4b-4384-90a1-7424923ee2e2 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.841873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cc0d-5041-467f-989b-46d66d53ced6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.844708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 805/1610] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:26.846002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ef5e-1e4b-4384-90a1-7424923ee2e2 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.846345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da128-2474-4783-87da-c1af379b4d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.847074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da128-2474-4783-87da-c1af379b4d5e service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.847316 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da128-2474-4783-87da-c1af379b4d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.847593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da128-2474-4783-87da-c1af379b4d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.856345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ef5e-1e4b-4384-90a1-7424923ee2e2 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.857681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093da128-2474-4783-87da-c1af379b4d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.859134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093da128-2474-4783-87da-c1af379b4d5e service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.861640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc5e98-1c19-43da-b7e6-b3ecb8aee7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.862635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc5e98-1c19-43da-b7e6-b3ecb8aee7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.863259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc5e98-1c19-43da-b7e6-b3ecb8aee7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.863537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc5e98-1c19-43da-b7e6-b3ecb8aee7d5 admin admin] PATH_INFO: `/v3/projects/c62d2cabbce24731812abe82635ad0f6/users/f053a8faa0be409888846405c993edd7/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.870604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 806/1611] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:26.871385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da128-2474-4783-87da-c1af379b4d5e service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.881797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc5e98-1c19-43da-b7e6-b3ecb8aee7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62d2cabbce24731812abe82635ad0f6, user_id=f053a8faa0be409888846405c993edd7, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.882494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da128-2474-4783-87da-c1af379b4d5e service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:26.884895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc5e98-1c19-43da-b7e6-b3ecb8aee7d5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.887932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75410dd-653d-4402-8ac7-c5ea6bec7104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:26.889211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75410dd-653d-4402-8ac7-c5ea6bec7104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.889477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75410dd-653d-4402-8ac7-c5ea6bec7104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.889741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75410dd-653d-4402-8ac7-c5ea6bec7104 admin admin] PATH_INFO: `/v3/projects/dca239d889d64af38ae3ba99fd9b41e7/users/afed68a2c1db4507b71ce0c850550937/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.905635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75410dd-653d-4402-8ac7-c5ea6bec7104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca239d889d64af38ae3ba99fd9b41e7, user_id=afed68a2c1db4507b71ce0c850550937, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:26.908985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75410dd-653d-4402-8ac7-c5ea6bec7104 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:26.983600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 807/1612] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:26 2022] PUT /identity/v3/projects/dca239d889d64af38ae3ba99fd9b41e7/users/afed68a2c1db4507b71ce0c850550937/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:26.983600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6dc7-fb6a-43a6-ab89-f06265b8ed18 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:26.983849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 806/1613] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:26 2022] PUT /identity/v3/projects/c62d2cabbce24731812abe82635ad0f6/users/f053a8faa0be409888846405c993edd7/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:26.983933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6dc7-fb6a-43a6-ab89-f06265b8ed18 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:26.983933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6dc7-fb6a-43a6-ab89-f06265b8ed18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:26.984130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc5e98-1c19-43da-b7e6-b3ecb8aee7d5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:26.994222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:27.024357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-954f6dc7-fb6a-43a6-ab89-f06265b8ed18 None None] MFA Rules not processed for user `b191071f184241908b1260b46c35a8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:27.025898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-954f6dc7-fb6a-43a6-ab89-f06265b8ed18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:27.072081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc5e98-1c19-43da-b7e6-b3ecb8aee7d5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:27.077963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dad5f-4121-4cd5-875c-b6b07d6a99e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:27.078783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dad5f-4121-4cd5-875c-b6b07d6a99e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.079164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dad5f-4121-4cd5-875c-b6b07d6a99e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.079614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dad5f-4121-4cd5-875c-b6b07d6a99e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.080991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dad5f-4121-4cd5-875c-b6b07d6a99e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:27.084905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dad5f-4121-4cd5-875c-b6b07d6a99e0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:27.100399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 807/1614] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:27.100399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17068a6e-0fc6-4b22-8328-97dd9d0571f8 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.100399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17068a6e-0fc6-4b22-8328-97dd9d0571f8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.100399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17068a6e-0fc6-4b22-8328-97dd9d0571f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.112490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 808/1615] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:27.123579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f6dc7-fb6a-43a6-ab89-f06265b8ed18 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:27.140131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-17068a6e-0fc6-4b22-8328-97dd9d0571f8 None None] MFA Rules not processed for user `57dd68408eb14eda9586e4d1283a563e`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:27.142320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-17068a6e-0fc6-4b22-8328-97dd9d0571f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:27.181741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f6dc7-fb6a-43a6-ab89-f06265b8ed18 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:27.188310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392b568-c7bf-459b-bf80-2711ba3e50c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:27.190040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392b568-c7bf-459b-bf80-2711ba3e50c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.190428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392b568-c7bf-459b-bf80-2711ba3e50c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.190855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392b568-c7bf-459b-bf80-2711ba3e50c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.197833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:27.202437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392b568-c7bf-459b-bf80-2711ba3e50c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:27.204622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392b568-c7bf-459b-bf80-2711ba3e50c0 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:27.221891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 808/1616] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:27.221891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b6d6e-9989-4fcd-8e0a-aa9e51fde368 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.221891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b6d6e-9989-4fcd-8e0a-aa9e51fde368 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.221891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b6d6e-9989-4fcd-8e0a-aa9e51fde368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.222382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 809/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:27.222382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281cb0f-3db5-467d-a7fb-0e9386de3e3c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.222914 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281cb0f-3db5-467d-a7fb-0e9386de3e3c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.223325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281cb0f-3db5-467d-a7fb-0e9386de3e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.253845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-317b6d6e-9989-4fcd-8e0a-aa9e51fde368 None None] MFA Rules not processed for user `f053a8faa0be409888846405c993edd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:27.255489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-317b6d6e-9989-4fcd-8e0a-aa9e51fde368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:27.258444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c281cb0f-3db5-467d-a7fb-0e9386de3e3c None None] MFA Rules not processed for user `afed68a2c1db4507b71ce0c850550937`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:27.259948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c281cb0f-3db5-467d-a7fb-0e9386de3e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:27.328024 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 809/1618] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:27.328872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b6d6e-9989-4fcd-8e0a-aa9e51fde368 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:27.340617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 810/1619] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:27.341686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b6d6e-9989-4fcd-8e0a-aa9e51fde368 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:27.342173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ca5a1-efa4-4077-a2f9-41691e1df2dc None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.342630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ca5a1-efa4-4077-a2f9-41691e1df2dc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.342853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ca5a1-efa4-4077-a2f9-41691e1df2dc None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.344613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 811/1620] 10.222.0.20 () {54 vars in 961 bytes} [Fri Jul 1 04:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:04:27.346289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896209fc-a9d1-45d8-905d-62899ee349f0 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.346289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896209fc-a9d1-45d8-905d-62899ee349f0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.346475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896209fc-a9d1-45d8-905d-62899ee349f0 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.351525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442bcf-0811-466b-86ab-d93120425341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:27.352594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 812/1621] 10.222.0.20 () {54 vars in 961 bytes} [Fri Jul 1 04:04:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:04:27.352717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442bcf-0811-466b-86ab-d93120425341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.352997 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442bcf-0811-466b-86ab-d93120425341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.353341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442bcf-0811-466b-86ab-d93120425341 admin admin] PATH_INFO: `/v3/projects/3aabfc8294b342dab1026cf8cd8b570a/users/93641feb18044278b072f9288d47c46b/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.353720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896209fc-a9d1-45d8-905d-62899ee349f0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:27.355834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442bcf-0811-466b-86ab-d93120425341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aabfc8294b342dab1026cf8cd8b570a, user_id=93641feb18044278b072f9288d47c46b, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:27.358522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442bcf-0811-466b-86ab-d93120425341 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:27.368455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896209fc-a9d1-45d8-905d-62899ee349f0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:27.373068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 810/1622] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:27 2022] PUT /identity/v3/projects/3aabfc8294b342dab1026cf8cd8b570a/users/93641feb18044278b072f9288d47c46b/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:27.373612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4fe63-2c50-47db-9c19-1a17969c820b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.373859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4fe63-2c50-47db-9c19-1a17969c820b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.374226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4fe63-2c50-47db-9c19-1a17969c820b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.389649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:27.398124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b7f4fe63-2c50-47db-9c19-1a17969c820b None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:27.399372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b7f4fe63-2c50-47db-9c19-1a17969c820b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:27.445980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4dc7-1bf5-4117-a18f-1a39ca18937f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:27.446659 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4dc7-1bf5-4117-a18f-1a39ca18937f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.446938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4dc7-1bf5-4117-a18f-1a39ca18937f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.447690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4dc7-1bf5-4117-a18f-1a39ca18937f admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.447933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb4dc7-1bf5-4117-a18f-1a39ca18937f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:27.449962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb4dc7-1bf5-4117-a18f-1a39ca18937f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:27.482022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 811/1623] 10.222.0.20 () {58 vars in 1058 bytes} [Fri Jul 1 04:04:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:27.483409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6e839-83f8-4d69-b100-d09d4a3203ed None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.483716 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6e839-83f8-4d69-b100-d09d4a3203ed None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.483965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6e839-83f8-4d69-b100-d09d4a3203ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.506072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-12b6e839-83f8-4d69-b100-d09d4a3203ed None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:27.507322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-12b6e839-83f8-4d69-b100-d09d4a3203ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:27.517342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 812/1624] 10.222.0.20 () {58 vars in 1058 bytes} [Fri Jul 1 04:04:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:27.518185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6e839-83f8-4d69-b100-d09d4a3203ed None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:27.558990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 813/1625] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:27 2022] POST /identity/v3/projects => generated 371 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:27.560651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5db13-dafc-4267-8699-5f94e8879a7a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.561918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5db13-dafc-4267-8699-5f94e8879a7a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.562310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5db13-dafc-4267-8699-5f94e8879a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.589033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6e839-83f8-4d69-b100-d09d4a3203ed None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:27.594679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9de5db13-dafc-4267-8699-5f94e8879a7a None None] MFA Rules not processed for user `93641feb18044278b072f9288d47c46b`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:27.596175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9de5db13-dafc-4267-8699-5f94e8879a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:27.646224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dcf9f-fed1-419a-b2a9-e3493c66a237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:27.647167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dcf9f-fed1-419a-b2a9-e3493c66a237 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.647167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dcf9f-fed1-419a-b2a9-e3493c66a237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.647449 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dcf9f-fed1-419a-b2a9-e3493c66a237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.676798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 814/1626] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:27.677856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5db13-dafc-4267-8699-5f94e8879a7a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:27.738749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dcf9f-fed1-419a-b2a9-e3493c66a237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:27.739846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dcf9f-fed1-419a-b2a9-e3493c66a237 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:27.762498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 813/1627] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:27.763454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dcf9f-fed1-419a-b2a9-e3493c66a237 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:27.792268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5db13-dafc-4267-8699-5f94e8879a7a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:27.798375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea35fa-35ff-4c27-a0cf-b946467923ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:27.799064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea35fa-35ff-4c27-a0cf-b946467923ae service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.799395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea35fa-35ff-4c27-a0cf-b946467923ae service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.799728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea35fa-35ff-4c27-a0cf-b946467923ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.883945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dcf9f-fed1-419a-b2a9-e3493c66a237 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:27.896717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d65049-c3a1-4c7a-ae85-2d8ceeb0a6e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:27.897394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d65049-c3a1-4c7a-ae85-2d8ceeb0a6e3 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:27.899035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d65049-c3a1-4c7a-ae85-2d8ceeb0a6e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:27.899347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d65049-c3a1-4c7a-ae85-2d8ceeb0a6e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:27.913056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea35fa-35ff-4c27-a0cf-b946467923ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:27.915511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea35fa-35ff-4c27-a0cf-b946467923ae service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:27.931708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 815/1628] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:27.932595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea35fa-35ff-4c27-a0cf-b946467923ae service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.039089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d65049-c3a1-4c7a-ae85-2d8ceeb0a6e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:28.043100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d65049-c3a1-4c7a-ae85-2d8ceeb0a6e3 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:28.055614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea35fa-35ff-4c27-a0cf-b946467923ae service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:28.059131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 814/1629] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:28.060186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d65049-c3a1-4c7a-ae85-2d8ceeb0a6e3 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.061553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823abb6-fdab-4131-a7aa-605bb66ccc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:28.062503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823abb6-fdab-4131-a7aa-605bb66ccc0e admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.062987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823abb6-fdab-4131-a7aa-605bb66ccc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.063458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823abb6-fdab-4131-a7aa-605bb66ccc0e admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.064838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823abb6-fdab-4131-a7aa-605bb66ccc0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:28.066933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823abb6-fdab-4131-a7aa-605bb66ccc0e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:28.072134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d65049-c3a1-4c7a-ae85-2d8ceeb0a6e3 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:28.076966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a490e31-5ff5-438a-93d2-52f38d8ee7f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:28.080797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a490e31-5ff5-438a-93d2-52f38d8ee7f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.080797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a490e31-5ff5-438a-93d2-52f38d8ee7f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.080797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a490e31-5ff5-438a-93d2-52f38d8ee7f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.091016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a490e31-5ff5-438a-93d2-52f38d8ee7f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:28.094118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a490e31-5ff5-438a-93d2-52f38d8ee7f0 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:28.110542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 815/1630] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:28.159999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a490e31-5ff5-438a-93d2-52f38d8ee7f0 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.179674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a490e31-5ff5-438a-93d2-52f38d8ee7f0 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:28.197625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:28.199180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.199498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.199777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] PATH_INFO: `/v3/users/2055155c6a7f49709124b2f00281c14f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.202518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2055155c6a7f49709124b2f00281c14f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:28.207055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 816/1631] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:27 2022] POST /identity/v3/users => generated 292 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:28.207897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:28.212225 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823abb6-fdab-4131-a7aa-605bb66ccc0e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.235458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823abb6-fdab-4131-a7aa-605bb66ccc0e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:28.255241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19936e3f-68c3-4742-a56b-e7a3200ad89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:28.256098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19936e3f-68c3-4742-a56b-e7a3200ad89b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.256472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19936e3f-68c3-4742-a56b-e7a3200ad89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.256865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19936e3f-68c3-4742-a56b-e7a3200ad89b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.257702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19936e3f-68c3-4742-a56b-e7a3200ad89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:28.259437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19936e3f-68c3-4742-a56b-e7a3200ad89b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:28.281614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 817/1632] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:28.285678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19936e3f-68c3-4742-a56b-e7a3200ad89b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.301439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:28.305994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19936e3f-68c3-4742-a56b-e7a3200ad89b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:28.318145 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cacd3-33c2-4239-a4b9-6ac2a49854eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:28.319427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cacd3-33c2-4239-a4b9-6ac2a49854eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.321039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cacd3-33c2-4239-a4b9-6ac2a49854eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.321039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cacd3-33c2-4239-a4b9-6ac2a49854eb admin admin] PATH_INFO: `/v3/projects/05ed553e46ac4ddf9a11912770c5f37e/users/bb0cebd8be534926aa178ca124b5a011/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.337525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cacd3-33c2-4239-a4b9-6ac2a49854eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=05ed553e46ac4ddf9a11912770c5f37e, user_id=bb0cebd8be534926aa178ca124b5a011, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:28.353405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cacd3-33c2-4239-a4b9-6ac2a49854eb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:28.445435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 818/1633] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:28 2022] PUT /identity/v3/projects/05ed553e46ac4ddf9a11912770c5f37e/users/bb0cebd8be534926aa178ca124b5a011/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:28.449726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ed3c-f36a-434d-a622-bed54c483fd1 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.450014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ed3c-f36a-434d-a622-bed54c483fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.450291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ed3c-f36a-434d-a622-bed54c483fd1 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.455661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 819/1634] 10.222.0.20 () {54 vars in 912 bytes} [Fri Jul 1 04:04:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:04:28.457878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5ed3c-f36a-434d-a622-bed54c483fd1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.464425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2055155c6a7f49709124b2f00281c14f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:28.466221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2055155c6a7f49709124b2f00281c14f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:28.466982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2055155c6a7f49709124b2f00281c14f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:28.488437 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2055155c6a7f49709124b2f00281c14f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:28.510978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d3239337-500b-45a8-b59a-01ba3db10589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2055155c6a7f49709124b2f00281c14f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:28.510978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:28.562846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5ed3c-f36a-434d-a622-bed54c483fd1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:28.568849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7abc0-8641-43d4-aecc-7e75af6492be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:28.569680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7abc0-8641-43d4-aecc-7e75af6492be admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.569960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7abc0-8641-43d4-aecc-7e75af6492be admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.570235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7abc0-8641-43d4-aecc-7e75af6492be admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.570969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7abc0-8641-43d4-aecc-7e75af6492be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:28.572601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7abc0-8641-43d4-aecc-7e75af6492be admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:28.583902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 820/1635] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:28.585658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a6cb2-4334-4923-97b3-957112fa19ac None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.585948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a6cb2-4334-4923-97b3-957112fa19ac None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.586227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a6cb2-4334-4923-97b3-957112fa19ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.606439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0d1a6cb2-4334-4923-97b3-957112fa19ac None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:28.607563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0d1a6cb2-4334-4923-97b3-957112fa19ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:28.620626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 816/1636] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:28 2022] DELETE /identity/v3/users/2055155c6a7f49709124b2f00281c14f => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:28.622231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766a69-0205-4b6a-8deb-314087613c61 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.622526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766a69-0205-4b6a-8deb-314087613c61 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.623359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766a69-0205-4b6a-8deb-314087613c61 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.629244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 817/1637] 10.222.0.20 () {54 vars in 912 bytes} [Fri Jul 1 04:04:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:04:28.630090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59766a69-0205-4b6a-8deb-314087613c61 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.631884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-59766a69-0205-4b6a-8deb-314087613c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:28.702627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 821/1638] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:04:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:28.711616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b45862-34ac-4aa7-b32e-ff758117b759 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.712203 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b45862-34ac-4aa7-b32e-ff758117b759 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.712728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b45862-34ac-4aa7-b32e-ff758117b759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.736060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-78b45862-34ac-4aa7-b32e-ff758117b759 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:28.737959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-78b45862-34ac-4aa7-b32e-ff758117b759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:28.747165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 822/1639] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:28.748972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a6e1-6b95-4c8e-ab03-d35ec61f8532 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.749265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a6e1-6b95-4c8e-ab03-d35ec61f8532 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.749654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a6e1-6b95-4c8e-ab03-d35ec61f8532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.752127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59766a69-0205-4b6a-8deb-314087613c61 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:28.760718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:28.762359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:28.762853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:28.763442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] PATH_INFO: `/v3/projects/05ed553e46ac4ddf9a11912770c5f37e/users/bb0cebd8be534926aa178ca124b5a011/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:28.766907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=05ed553e46ac4ddf9a11912770c5f37e, user_id=bb0cebd8be534926aa178ca124b5a011, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:28.770775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:28.791550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ab41a6e1-6b95-4c8e-ab03-d35ec61f8532 None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:28.797383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ab41a6e1-6b95-4c8e-ab03-d35ec61f8532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:28.804836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 818/1640] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:28 2022] PUT /identity/v3/projects/05ed553e46ac4ddf9a11912770c5f37e/users/bb0cebd8be534926aa178ca124b5a011/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:28.805753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.824190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:28.825417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:28.973573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 823/1641] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:04:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:28.976693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:28.978464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41a6e1-6b95-4c8e-ab03-d35ec61f8532 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:28.980348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ab41a6e1-6b95-4c8e-ab03-d35ec61f8532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:28.984491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca84b4-9337-488a-8881-4247a8017ecf admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:28.997396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59185080-dfbd-479a-9dba-f8457418f959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:29.000570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59185080-dfbd-479a-9dba-f8457418f959 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:29.002344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59185080-dfbd-479a-9dba-f8457418f959 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:29.004153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59185080-dfbd-479a-9dba-f8457418f959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:29.039738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59185080-dfbd-479a-9dba-f8457418f959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:29.046094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59185080-dfbd-479a-9dba-f8457418f959 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:29.090281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 819/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:29.091732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd9ff-dab1-47f2-a389-3e13ba5ed4b7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:29.092019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd9ff-dab1-47f2-a389-3e13ba5ed4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:29.092298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd9ff-dab1-47f2-a389-3e13ba5ed4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:29.161893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3e9fd9ff-dab1-47f2-a389-3e13ba5ed4b7 None None] MFA Rules not processed for user `bb0cebd8be534926aa178ca124b5a011`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:29.164339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3e9fd9ff-dab1-47f2-a389-3e13ba5ed4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:29.165350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41a6e1-6b95-4c8e-ab03-d35ec61f8532 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:29.173193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e6f05-e16b-4934-8bb2-a7329cf0134b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:29.174486 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e6f05-e16b-4934-8bb2-a7329cf0134b service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:29.174800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e6f05-e16b-4934-8bb2-a7329cf0134b service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:29.175076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e6f05-e16b-4934-8bb2-a7329cf0134b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:29.315843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 820/1643] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:29.315843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fd9ff-dab1-47f2-a389-3e13ba5ed4b7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:29.315843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3e9fd9ff-dab1-47f2-a389-3e13ba5ed4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:29.339522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fd9ff-dab1-47f2-a389-3e13ba5ed4b7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:29.346780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e6f05-e16b-4934-8bb2-a7329cf0134b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:29.349928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e6f05-e16b-4934-8bb2-a7329cf0134b service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:29.356437 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:29.359077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:29.367904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:29.372242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:29.381466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 824/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:29.429798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:29.430541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:29.499418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 821/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:29.546474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:30.362391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e6f05-e16b-4934-8bb2-a7329cf0134b service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:30.373386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e6f05-e16b-4934-8bb2-a7329cf0134b service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:30.379104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e5b1f-0b99-4e34-b237-2b40061c1164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:30.380124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5b1f-0b99-4e34-b237-2b40061c1164 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:30.380627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5b1f-0b99-4e34-b237-2b40061c1164 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:30.381112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5b1f-0b99-4e34-b237-2b40061c1164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:30.392779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e5b1f-0b99-4e34-b237-2b40061c1164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:30.395140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e5b1f-0b99-4e34-b237-2b40061c1164 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:30.424109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 825/1646] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:31.111550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:31.114228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:31.138503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46c61d-45bd-4eb7-9aa1-d7e28738cd16 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:31.146398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:31.149005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:31.149766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:31.150986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] PATH_INFO: `/v3/projects/cbc23d0f41844c50af19f78ee6bf409d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:31.171293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc23d0f41844c50af19f78ee6bf409d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:31.171293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:04:31.171293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 01 04:04:31.171293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: Jul 01 04:04:31.171293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:31.172277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:31.346773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc23d0f41844c50af19f78ee6bf409d', 'request_id': 'req-eb39aecf-cb76-4b29-b206-ddd098c09a05'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:31.355478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc23d0f41844c50af19f78ee6bf409d', 'request_id': 'req-eb39aecf-cb76-4b29-b206-ddd098c09a05'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:31.416433 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc23d0f41844c50af19f78ee6bf409d', 'request_id': 'req-eb39aecf-cb76-4b29-b206-ddd098c09a05'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:31.485459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] The token cache is being invalidate because project cbc23d0f41844c50af19f78ee6bf409d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:31.486182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:31.491301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:31.496467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 822/1647] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:31 2022] DELETE /identity/v3/projects/cbc23d0f41844c50af19f78ee6bf409d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:31.504080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e5b1f-0b99-4e34-b237-2b40061c1164 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:31.524066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e5b1f-0b99-4e34-b237-2b40061c1164 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:31.737982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:31.750344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:31.750344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:31.750344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] PATH_INFO: `/v3/users/93641feb18044278b072f9288d47c46b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:31.750344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93641feb18044278b072f9288d47c46b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:31.750344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 04:04:31.750344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 01 04:04:31.750344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: Jul 01 04:04:31.754627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:32.184750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:32.212893 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93641feb18044278b072f9288d47c46b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:32.213716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93641feb18044278b072f9288d47c46b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:32.303606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93641feb18044278b072f9288d47c46b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:32.342052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93641feb18044278b072f9288d47c46b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:32.343002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93641feb18044278b072f9288d47c46b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:32.392922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 826/1648] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:31 2022] DELETE /identity/v3/users/93641feb18044278b072f9288d47c46b => generated 0 bytes in 889 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:33.480078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:33.482177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:33.509950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39aecf-cb76-4b29-b206-ddd098c09a05 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:33.764954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:33.765828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:33.766180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:33.766467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] PATH_INFO: `/v3/users/f053a8faa0be409888846405c993edd7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:33.766710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:33.774213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:33.774454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f053a8faa0be409888846405c993edd7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:33.777063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:33.835152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08427fc-442b-4130-940b-7b008929486c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:33.853187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:33.854314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:33.854756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:33.864105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] PATH_INFO: `/v3/projects/3aabfc8294b342dab1026cf8cd8b570a` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:33.873021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aabfc8294b342dab1026cf8cd8b570a)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:33.878061 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:34.101628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f053a8faa0be409888846405c993edd7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.103852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f053a8faa0be409888846405c993edd7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.106223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f053a8faa0be409888846405c993edd7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.128578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aabfc8294b342dab1026cf8cd8b570a', 'request_id': 'req-e2364247-c219-4341-97c7-1394ed9ed709'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.172391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f053a8faa0be409888846405c993edd7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.172781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aabfc8294b342dab1026cf8cd8b570a', 'request_id': 'req-e2364247-c219-4341-97c7-1394ed9ed709'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.208682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a64ffb2e-5261-4137-b714-225dd50e5ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f053a8faa0be409888846405c993edd7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.212884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aabfc8294b342dab1026cf8cd8b570a', 'request_id': 'req-e2364247-c219-4341-97c7-1394ed9ed709'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.308751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 823/1649] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:33 2022] DELETE /identity/v3/users/f053a8faa0be409888846405c993edd7 => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:34.316651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794a5b7-0f36-4026-9fb0-c7406c1f1c62 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:34.317682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794a5b7-0f36-4026-9fb0-c7406c1f1c62 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:34.319334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794a5b7-0f36-4026-9fb0-c7406c1f1c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:34.363330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e794a5b7-0f36-4026-9fb0-c7406c1f1c62 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:34.365892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e794a5b7-0f36-4026-9fb0-c7406c1f1c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:34.388870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:34.396472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] The token cache is being invalidate because project 3aabfc8294b342dab1026cf8cd8b570a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:34.396782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.398234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e2364247-c219-4341-97c7-1394ed9ed709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:34.403207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 827/1650] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:33 2022] DELETE /identity/v3/projects/3aabfc8294b342dab1026cf8cd8b570a => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:34.410770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3b46-7e16-42ad-a358-20f3c384539b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:34.416151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3b46-7e16-42ad-a358-20f3c384539b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:34.416151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3b46-7e16-42ad-a358-20f3c384539b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:34.462646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f81a3b46-7e16-42ad-a358-20f3c384539b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:34.464090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f81a3b46-7e16-42ad-a358-20f3c384539b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:34.621122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 828/1651] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:04:34.624154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 824/1652] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:04:34.630570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720d589-e782-46c9-9a98-407c038200e4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:34.631127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720d589-e782-46c9-9a98-407c038200e4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:34.631571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720d589-e782-46c9-9a98-407c038200e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:34.656847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794a5b7-0f36-4026-9fb0-c7406c1f1c62 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:34.659819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e794a5b7-0f36-4026-9fb0-c7406c1f1c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:34.707710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7720d589-e782-46c9-9a98-407c038200e4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:34.707710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7720d589-e782-46c9-9a98-407c038200e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:34.745071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 829/1653] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:04:34.746363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720d589-e782-46c9-9a98-407c038200e4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:34.748279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7720d589-e782-46c9-9a98-407c038200e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:34.765959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720d589-e782-46c9-9a98-407c038200e4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:34.773921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:34.776230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:34.776730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:34.777474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] PATH_INFO: `/v3/users/b191071f184241908b1260b46c35a8ce` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:34.780066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b191071f184241908b1260b46c35a8ce)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:34.797050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:34.809764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794a5b7-0f36-4026-9fb0-c7406c1f1c62 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:34.827772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce86d3b-1ba6-4c88-adc2-b06a0fb94578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:34.829440 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce86d3b-1ba6-4c88-adc2-b06a0fb94578 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:34.830485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce86d3b-1ba6-4c88-adc2-b06a0fb94578 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:34.830940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce86d3b-1ba6-4c88-adc2-b06a0fb94578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:34.850825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce86d3b-1ba6-4c88-adc2-b06a0fb94578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:34.854947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce86d3b-1ba6-4c88-adc2-b06a0fb94578 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:34.896427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 825/1654] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:04:34.898652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce86d3b-1ba6-4c88-adc2-b06a0fb94578 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:34.928654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce86d3b-1ba6-4c88-adc2-b06a0fb94578 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:34.955292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:34.955292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:34.958610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:34.959042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:34.974376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:34.976311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:34.985275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:34.991566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:35.035819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 826/1655] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:35.037545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:35.079309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f863e-69e0-4d03-ac90-3e4e67fe4d5e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:35.113294 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:35.114046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:35.114337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:35.114610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:35.115354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:35.118867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:35.123149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b191071f184241908b1260b46c35a8ce'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:35.123950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b191071f184241908b1260b46c35a8ce'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:35.198620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b191071f184241908b1260b46c35a8ce'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:35.212018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b191071f184241908b1260b46c35a8ce'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:35.213095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57c2eb9-f193-4dc6-80cc-35a598375f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b191071f184241908b1260b46c35a8ce'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:35.273193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 827/1656] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:35 2022] POST /identity/v3/projects => generated 385 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:35.275832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:35.276115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 830/1657] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:34 2022] DELETE /identity/v3/users/b191071f184241908b1260b46c35a8ce => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:35.284452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224c984-098d-422b-9011-5cbd59805db5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:35.286470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224c984-098d-422b-9011-5cbd59805db5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:35.286470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224c984-098d-422b-9011-5cbd59805db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:35.289424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:35.312608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e224c984-098d-422b-9011-5cbd59805db5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:35.316369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e224c984-098d-422b-9011-5cbd59805db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:35.433567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 831/1658] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:35.502897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224c984-098d-422b-9011-5cbd59805db5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:35.504550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e224c984-098d-422b-9011-5cbd59805db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:35.506489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32e383-9bc9-4bc6-a798-e8d0503db6de admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:35.529614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3cb6e-894d-40b9-b41c-a38916836ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:35.530333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3cb6e-894d-40b9-b41c-a38916836ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:35.530616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3cb6e-894d-40b9-b41c-a38916836ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:35.530887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3cb6e-894d-40b9-b41c-a38916836ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:35.531626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3cb6e-894d-40b9-b41c-a38916836ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:35.534753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3cb6e-894d-40b9-b41c-a38916836ec9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:35.679118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 828/1659] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:35 2022] POST /identity/v3/users => generated 293 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:35.681206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224c984-098d-422b-9011-5cbd59805db5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:35.684050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3cb6e-894d-40b9-b41c-a38916836ec9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:35.687703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec3df8-237c-485b-956d-a3628d88e557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:35.688386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3df8-237c-485b-956d-a3628d88e557 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:35.688668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3df8-237c-485b-956d-a3628d88e557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:35.689004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3df8-237c-485b-956d-a3628d88e557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:35.707353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3cb6e-894d-40b9-b41c-a38916836ec9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:35.732402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec3df8-237c-485b-956d-a3628d88e557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:35.733864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec3df8-237c-485b-956d-a3628d88e557 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:35.747453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bbf07-2593-4cbe-8cff-8a9bf67fc6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:35.750557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bbf07-2593-4cbe-8cff-8a9bf67fc6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:35.753669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bbf07-2593-4cbe-8cff-8a9bf67fc6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:35.753669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bbf07-2593-4cbe-8cff-8a9bf67fc6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:35.766127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bbf07-2593-4cbe-8cff-8a9bf67fc6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:35.785547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bbf07-2593-4cbe-8cff-8a9bf67fc6e2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:35.811059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 829/1660] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:35.815604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bbf07-2593-4cbe-8cff-8a9bf67fc6e2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:35.819498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 832/1661] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:35.845042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bbf07-2593-4cbe-8cff-8a9bf67fc6e2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:35.905140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059c771-999e-4b53-a576-d604ad1d441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:35.905856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059c771-999e-4b53-a576-d604ad1d441c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:35.906286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059c771-999e-4b53-a576-d604ad1d441c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:35.906575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059c771-999e-4b53-a576-d604ad1d441c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:35.907431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059c771-999e-4b53-a576-d604ad1d441c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:35.909943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059c771-999e-4b53-a576-d604ad1d441c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:35.940705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 830/1662] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:35.946212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec3df8-237c-485b-956d-a3628d88e557 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:35.976738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec3df8-237c-485b-956d-a3628d88e557 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:36.014539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a0fd-aed4-4364-9f56-23cdc83eaf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:36.019270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a0fd-aed4-4364-9f56-23cdc83eaf95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:36.019270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a0fd-aed4-4364-9f56-23cdc83eaf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:36.023304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a0fd-aed4-4364-9f56-23cdc83eaf95 admin admin] PATH_INFO: `/v3/projects/7e313ea341f54449b01ac438c507885c/users/36513c96c5a0486c86ccf8474cc12bb4/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:36.134218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a0fd-aed4-4364-9f56-23cdc83eaf95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e313ea341f54449b01ac438c507885c, user_id=36513c96c5a0486c86ccf8474cc12bb4, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:36.150503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a0fd-aed4-4364-9f56-23cdc83eaf95 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:36.356278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 833/1663] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:35 2022] PUT /identity/v3/projects/7e313ea341f54449b01ac438c507885c/users/36513c96c5a0486c86ccf8474cc12bb4/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:36.359217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064c52-2d97-4af6-986b-b751d0397a46 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:36.359598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064c52-2d97-4af6-986b-b751d0397a46 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:36.360054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064c52-2d97-4af6-986b-b751d0397a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:36.407411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6c064c52-2d97-4af6-986b-b751d0397a46 None None] MFA Rules not processed for user `36513c96c5a0486c86ccf8474cc12bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:36.408740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6c064c52-2d97-4af6-986b-b751d0397a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:36.492386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:36.526536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 831/1664] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:36.530744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a0fd-aed4-4364-9f56-23cdc83eaf95 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:36.696618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a0fd-aed4-4364-9f56-23cdc83eaf95 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:36.728744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:36.729974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:36.730479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:36.731311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:36.734251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:36.740002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:36.894338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 834/1665] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:36 2022] POST /identity/v3/projects => generated 387 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:36.898428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064c52-2d97-4af6-986b-b751d0397a46 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:36.922746 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064c52-2d97-4af6-986b-b751d0397a46 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:37.004575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:37.006312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:37.056369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:37.057041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:37.057483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:37.057873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:37.058749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:37.073568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:37.153443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0813c-98ee-4ca0-9a78-f6829464674e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:37.166627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 832/1666] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:36 2022] POST /identity/v3/users => generated 300 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:37.173106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:37.175931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:37.182615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:37.183550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:37.183837 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:37.184185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] PATH_INFO: `/v3/projects/74f993901b2b497d904ad1e4343954da` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:37.186566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f993901b2b497d904ad1e4343954da)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:37.188599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:37.195433 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875fa62-cea1-4fa7-8240-54c6ffce40b8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:37.201778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:37.201778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:37.202083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:37.202387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] PATH_INFO: `/v3/projects/c62d2cabbce24731812abe82635ad0f6` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:37.206763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62d2cabbce24731812abe82635ad0f6)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:37.214045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:37.289612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:37.377555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f993901b2b497d904ad1e4343954da', 'request_id': 'req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.393939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f993901b2b497d904ad1e4343954da', 'request_id': 'req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.409904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f993901b2b497d904ad1e4343954da', 'request_id': 'req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.500563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62d2cabbce24731812abe82635ad0f6', 'request_id': 'req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.526335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62d2cabbce24731812abe82635ad0f6', 'request_id': 'req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.538247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] The token cache is being invalidate because project 74f993901b2b497d904ad1e4343954da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:37.538247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.539704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.549294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 835/1667] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:37 2022] DELETE /identity/v3/projects/74f993901b2b497d904ad1e4343954da => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:04:37.550198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:37.553936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:37.649313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62d2cabbce24731812abe82635ad0f6', 'request_id': 'req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.685024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5bfd1-7e0c-45bd-94df-34cca34da1a3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:37.734324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] The token cache is being invalidate because project c62d2cabbce24731812abe82635ad0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:37.734324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.736117 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8dcf81b3-b6d1-4c34-968c-d1c5c9c15de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:37.747044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 833/1668] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:37 2022] DELETE /identity/v3/projects/c62d2cabbce24731812abe82635ad0f6 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:37.750186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44e4f-8526-4e37-ad28-8bdb17af3847 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:37.750890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44e4f-8526-4e37-ad28-8bdb17af3847 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:37.755917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44e4f-8526-4e37-ad28-8bdb17af3847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:37.814455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-65d44e4f-8526-4e37-ad28-8bdb17af3847 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:37.817150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-65d44e4f-8526-4e37-ad28-8bdb17af3847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:37.912791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:37.915097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:37.915990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:37.916769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:37.919726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:37.927321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:37.934213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:37.981120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 836/1669] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:37.982498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:37.992377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:38.040465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c267d26-a4f0-4ebf-8a91-f7a202ad01a4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:38.061791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 834/1670] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:38.067736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d44e4f-8526-4e37-ad28-8bdb17af3847 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:38.075125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-65d44e4f-8526-4e37-ad28-8bdb17af3847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:38.077554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:38.083668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:38.083999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:38.084370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] PATH_INFO: `/v3/users/bb0cebd8be534926aa178ca124b5a011` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:38.085851 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0cebd8be534926aa178ca124b5a011)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:38.088624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:38.169205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d44e4f-8526-4e37-ad28-8bdb17af3847 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:38.192438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5d30f-3afa-4d51-b45b-9f32d56d839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:38.203957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5d30f-3afa-4d51-b45b-9f32d56d839f admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:38.204510 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5d30f-3afa-4d51-b45b-9f32d56d839f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:38.204928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5d30f-3afa-4d51-b45b-9f32d56d839f admin admin] PATH_INFO: `/v3/projects/b0024ecc55604d69be1d2a5dca5af9cd/users/2d13e03a9b7c442391b9f4600966c4e4/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:38.232611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5d30f-3afa-4d51-b45b-9f32d56d839f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0024ecc55604d69be1d2a5dca5af9cd, user_id=2d13e03a9b7c442391b9f4600966c4e4, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:38.236540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5d30f-3afa-4d51-b45b-9f32d56d839f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:38.322520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0cebd8be534926aa178ca124b5a011'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:38.325100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0cebd8be534926aa178ca124b5a011'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:38.371784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 835/1671] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:38 2022] PUT /identity/v3/projects/b0024ecc55604d69be1d2a5dca5af9cd/users/2d13e03a9b7c442391b9f4600966c4e4/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:38.373419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b976d-a945-4ac6-be69-6331aaaf6645 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:38.373691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b976d-a945-4ac6-be69-6331aaaf6645 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:38.373970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b976d-a945-4ac6-be69-6331aaaf6645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:38.414907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1e4b976d-a945-4ac6-be69-6331aaaf6645 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:38.415973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1e4b976d-a945-4ac6-be69-6331aaaf6645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:38.462875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0cebd8be534926aa178ca124b5a011'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:38.476779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0cebd8be534926aa178ca124b5a011'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:38.479255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0cebd8be534926aa178ca124b5a011'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:38.542116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 837/1672] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:37 2022] DELETE /identity/v3/users/bb0cebd8be534926aa178ca124b5a011 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:38.543797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:38.549592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:38.613813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 836/1673] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:38.615250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b976d-a945-4ac6-be69-6331aaaf6645 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:38.637292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b976d-a945-4ac6-be69-6331aaaf6645 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:38.813560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:38.815094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:38.815562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:38.815992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:38.816916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:38.817376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:38.819504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:38.855671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 837/1674] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 04:04:38.856897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:38.879910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475d5c4-ad72-431d-9194-61555081b9e8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:38.897521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f10ed-7069-491a-a800-733033abb8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:38.905285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f10ed-7069-491a-a800-733033abb8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:38.905285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f10ed-7069-491a-a800-733033abb8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:38.905285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f10ed-7069-491a-a800-733033abb8e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:38.905285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f10ed-7069-491a-a800-733033abb8e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:38.905285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f10ed-7069-491a-a800-733033abb8e2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:39.039331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4458e7-4840-4bb2-8f76-896e912b396c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:39.072909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbc25a-fd2c-44aa-88e7-7e2bcf30f5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:39.075613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbc25a-fd2c-44aa-88e7-7e2bcf30f5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.076240 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 838/1675] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:38 2022] POST /identity/v3/projects => generated 365 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:39.080297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbc25a-fd2c-44aa-88e7-7e2bcf30f5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.080869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbc25a-fd2c-44aa-88e7-7e2bcf30f5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.081766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f10ed-7069-491a-a800-733033abb8e2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:39.086902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbc25a-fd2c-44aa-88e7-7e2bcf30f5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:39.103150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbc25a-fd2c-44aa-88e7-7e2bcf30f5fa admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:39.184551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 838/1676] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 638 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:39.187724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbc25a-fd2c-44aa-88e7-7e2bcf30f5fa admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:39.230045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbc25a-fd2c-44aa-88e7-7e2bcf30f5fa admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:39.235458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a656ff3-5660-49c7-a2f2-6bca14db77d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:39.236476 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a656ff3-5660-49c7-a2f2-6bca14db77d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.236695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a656ff3-5660-49c7-a2f2-6bca14db77d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.236980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a656ff3-5660-49c7-a2f2-6bca14db77d1 admin admin] PATH_INFO: `/v3/projects/b0024ecc55604d69be1d2a5dca5af9cd/users/2d13e03a9b7c442391b9f4600966c4e4/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.240806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a656ff3-5660-49c7-a2f2-6bca14db77d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0024ecc55604d69be1d2a5dca5af9cd, user_id=2d13e03a9b7c442391b9f4600966c4e4, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:39.244023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f10ed-7069-491a-a800-733033abb8e2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:39.247292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a656ff3-5660-49c7-a2f2-6bca14db77d1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:39.265675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12df65-9986-404d-bfb3-071c4ec22b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:39.268022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12df65-9986-404d-bfb3-071c4ec22b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.270253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12df65-9986-404d-bfb3-071c4ec22b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.270253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12df65-9986-404d-bfb3-071c4ec22b57 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.270253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12df65-9986-404d-bfb3-071c4ec22b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:39.272731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12df65-9986-404d-bfb3-071c4ec22b57 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:39.304129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 839/1677] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:39 2022] PUT /identity/v3/projects/b0024ecc55604d69be1d2a5dca5af9cd/users/2d13e03a9b7c442391b9f4600966c4e4/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:39.305948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36de9d-5c22-443e-b7a6-60504c4598f9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.306354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36de9d-5c22-443e-b7a6-60504c4598f9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.306702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36de9d-5c22-443e-b7a6-60504c4598f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.381686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7a36de9d-5c22-443e-b7a6-60504c4598f9 None None] MFA Rules not processed for user `2d13e03a9b7c442391b9f4600966c4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:39.401115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 839/1678] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:39 2022] POST /identity/v3/users => generated 283 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:39.402912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12df65-9986-404d-bfb3-071c4ec22b57 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:39.404660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7a36de9d-5c22-443e-b7a6-60504c4598f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:39.448998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12df65-9986-404d-bfb3-071c4ec22b57 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:39.579932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:39.624484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 840/1679] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:39.626217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd406-e51d-4b62-9c30-49a4b638211e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.628879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd406-e51d-4b62-9c30-49a4b638211e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.629332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd406-e51d-4b62-9c30-49a4b638211e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.689473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f9ddd406-e51d-4b62-9c30-49a4b638211e None None] MFA Rules not processed for user `8622ec5212c84d87b9b7c8f725012a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:39.691064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f9ddd406-e51d-4b62-9c30-49a4b638211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:39.710586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:39.711816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.714795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.715343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.720832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:39.725317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:39.740301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 840/1680] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:39.741576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:39.744105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:39.848928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:39.854781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 841/1681] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:39.865327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddd406-e51d-4b62-9c30-49a4b638211e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:39.891261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283e050-81e8-4c9a-ab30-24cb1e9fc8c1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:39.896885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec39b14-5474-4835-b465-e86b421aabdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:39.897961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec39b14-5474-4835-b465-e86b421aabdf service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.898735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec39b14-5474-4835-b465-e86b421aabdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.899542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec39b14-5474-4835-b465-e86b421aabdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.911547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddd406-e51d-4b62-9c30-49a4b638211e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:39.926205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a311f6-7a39-4ebc-bc0a-d9321ac8d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:39.928907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a311f6-7a39-4ebc-bc0a-d9321ac8d489 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.931905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec39b14-5474-4835-b465-e86b421aabdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:39.933956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a311f6-7a39-4ebc-bc0a-d9321ac8d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.934767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec39b14-5474-4835-b465-e86b421aabdf service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:39.935071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a311f6-7a39-4ebc-bc0a-d9321ac8d489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.936460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a311f6-7a39-4ebc-bc0a-d9321ac8d489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:39.939540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a311f6-7a39-4ebc-bc0a-d9321ac8d489 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:39.950630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 841/1682] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:39.954722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180ce43-6d3e-47e9-b637-b86de8e9126c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:39.957259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180ce43-6d3e-47e9-b637-b86de8e9126c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:39.958044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180ce43-6d3e-47e9-b637-b86de8e9126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:39.986623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 842/1683] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:39.991964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a311f6-7a39-4ebc-bc0a-d9321ac8d489 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:40.015071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b180ce43-6d3e-47e9-b637-b86de8e9126c None None] MFA Rules not processed for user `8622ec5212c84d87b9b7c8f725012a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:40.028500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b180ce43-6d3e-47e9-b637-b86de8e9126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:40.052952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a311f6-7a39-4ebc-bc0a-d9321ac8d489 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:40.066917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 842/1684] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:40.076657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767cd93-f29d-42d3-b338-8be2a4681203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:40.077591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767cd93-f29d-42d3-b338-8be2a4681203 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:40.077857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767cd93-f29d-42d3-b338-8be2a4681203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:40.078136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767cd93-f29d-42d3-b338-8be2a4681203 admin admin] PATH_INFO: `/v3/projects/96cd59ca92d64483b4eafb41f68a2b9f/users/7947bb65ecad43e89376ce6afa100d22/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:40.091000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180ce43-6d3e-47e9-b637-b86de8e9126c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:40.097031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b180ce43-6d3e-47e9-b637-b86de8e9126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:40.131261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767cd93-f29d-42d3-b338-8be2a4681203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96cd59ca92d64483b4eafb41f68a2b9f, user_id=7947bb65ecad43e89376ce6afa100d22, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:40.141205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767cd93-f29d-42d3-b338-8be2a4681203 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:40.250987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 843/1685] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:39 2022] PUT /identity/v3/projects/96cd59ca92d64483b4eafb41f68a2b9f/users/7947bb65ecad43e89376ce6afa100d22/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:40.252771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767cd93-f29d-42d3-b338-8be2a4681203 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:40.384334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180ce43-6d3e-47e9-b637-b86de8e9126c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:40.427525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767cd93-f29d-42d3-b338-8be2a4681203 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:40.433814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:40.434884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:40.439978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:40.439978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] PATH_INFO: `/v3/users/36513c96c5a0486c86ccf8474cc12bb4` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:40.439978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36513c96c5a0486c86ccf8474cc12bb4)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:40.439978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:40.497838 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc557c-fad1-42aa-bd6e-c66731fb67d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:40.498528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc557c-fad1-42aa-bd6e-c66731fb67d9 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:40.498862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc557c-fad1-42aa-bd6e-c66731fb67d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:40.499143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc557c-fad1-42aa-bd6e-c66731fb67d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:40.653060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc557c-fad1-42aa-bd6e-c66731fb67d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:40.654791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc557c-fad1-42aa-bd6e-c66731fb67d9 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:40.669928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 843/1686] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:04:40.671849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15effb63-09d7-4d50-859a-7bfb7e1196f4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:40.672237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15effb63-09d7-4d50-859a-7bfb7e1196f4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:40.672406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15effb63-09d7-4d50-859a-7bfb7e1196f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:40.748200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-15effb63-09d7-4d50-859a-7bfb7e1196f4 None None] MFA Rules not processed for user `7947bb65ecad43e89376ce6afa100d22`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:40.749825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-15effb63-09d7-4d50-859a-7bfb7e1196f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:40.778467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36513c96c5a0486c86ccf8474cc12bb4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:40.780253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36513c96c5a0486c86ccf8474cc12bb4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:40.883801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36513c96c5a0486c86ccf8474cc12bb4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:40.893445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36513c96c5a0486c86ccf8474cc12bb4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:40.894681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36513c96c5a0486c86ccf8474cc12bb4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:40.957597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 844/1687] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:40.962243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15effb63-09d7-4d50-859a-7bfb7e1196f4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:40.964439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-15effb63-09d7-4d50-859a-7bfb7e1196f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:40.976727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 844/1688] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:40 2022] DELETE /identity/v3/users/36513c96c5a0486c86ccf8474cc12bb4 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:40.978053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:40.982077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:40.984843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15effb63-09d7-4d50-859a-7bfb7e1196f4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:41.076485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:41.077607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:41.078865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:41.080139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] PATH_INFO: `/v3/projects/05ed553e46ac4ddf9a11912770c5f37e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:41.082128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05ed553e46ac4ddf9a11912770c5f37e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:41.084412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:41.093840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:41.191490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5ce17-358d-4de0-acf9-eeed8277e3f7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:41.200991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dd761-a693-4152-a4b0-aa05ddf47983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:41.202177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd761-a693-4152-a4b0-aa05ddf47983 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:41.202680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd761-a693-4152-a4b0-aa05ddf47983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:41.203557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd761-a693-4152-a4b0-aa05ddf47983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:41.205096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dd761-a693-4152-a4b0-aa05ddf47983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:41.209620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dd761-a693-4152-a4b0-aa05ddf47983 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:41.248608 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05ed553e46ac4ddf9a11912770c5f37e', 'request_id': 'req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:41.257366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05ed553e46ac4ddf9a11912770c5f37e', 'request_id': 'req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:41.334378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05ed553e46ac4ddf9a11912770c5f37e', 'request_id': 'req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:41.355376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 845/1689] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:40 2022] POST /identity/v3/projects => generated 365 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:41.356853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389485f-5662-439a-9483-b9392d117216 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:41.358305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389485f-5662-439a-9483-b9392d117216 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:41.358666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389485f-5662-439a-9483-b9392d117216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:41.380756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] The token cache is being invalidate because project 05ed553e46ac4ddf9a11912770c5f37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:41.380756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:41.386991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a8bedfc-57f0-4020-90e3-e81b21ec1aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:41.392429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 845/1690] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:40 2022] DELETE /identity/v3/projects/05ed553e46ac4ddf9a11912770c5f37e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:41.398634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0c48d-7e30-4e88-9cf8-12527cfd9883 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:41.399149 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0c48d-7e30-4e88-9cf8-12527cfd9883 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:41.400313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0c48d-7e30-4e88-9cf8-12527cfd9883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:41.400426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d389485f-5662-439a-9483-b9392d117216 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:41.401272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d389485f-5662-439a-9483-b9392d117216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:41.468517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d9f0c48d-7e30-4e88-9cf8-12527cfd9883 None None] MFA Rules not processed for user `8ab2da92b7e94592b80ef3d7852f51ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:41.470483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d9f0c48d-7e30-4e88-9cf8-12527cfd9883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:41.525910 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 846/1691] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:41.532305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389485f-5662-439a-9483-b9392d117216 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:41.542406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d389485f-5662-439a-9483-b9392d117216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:41.563897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:41.572953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389485f-5662-439a-9483-b9392d117216 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:41.585743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266578d-c51b-4dab-9908-25ce0fc7de33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:41.587518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266578d-c51b-4dab-9908-25ce0fc7de33 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:41.589077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266578d-c51b-4dab-9908-25ce0fc7de33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:41.589512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266578d-c51b-4dab-9908-25ce0fc7de33 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:41.590724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266578d-c51b-4dab-9908-25ce0fc7de33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:41.596139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266578d-c51b-4dab-9908-25ce0fc7de33 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:41.639148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 846/1692] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:41.640681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0c48d-7e30-4e88-9cf8-12527cfd9883 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:41.642855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d9f0c48d-7e30-4e88-9cf8-12527cfd9883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:41.662245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0c48d-7e30-4e88-9cf8-12527cfd9883 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:41.668201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:41.670981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:41.670981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:41.670981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] PATH_INFO: `/v3/users/57dd68408eb14eda9586e4d1283a563e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:41.670981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57dd68408eb14eda9586e4d1283a563e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:41.673237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:41.763638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 847/1693] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:41 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:41.765801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f2459-5bb9-4456-b695-ebeed80205ec None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:41.766106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f2459-5bb9-4456-b695-ebeed80205ec None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:41.766704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f2459-5bb9-4456-b695-ebeed80205ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:41.817388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f38f2459-5bb9-4456-b695-ebeed80205ec None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:41.819016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f38f2459-5bb9-4456-b695-ebeed80205ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:41.850534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 848/1694] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:41.850534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f2459-5bb9-4456-b695-ebeed80205ec None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:41.857941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f38f2459-5bb9-4456-b695-ebeed80205ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:41.955271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f2459-5bb9-4456-b695-ebeed80205ec None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:41.961701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74292d-f0cd-45ef-91c3-88b66d5ee8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:41.962391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74292d-f0cd-45ef-91c3-88b66d5ee8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:41.967905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74292d-f0cd-45ef-91c3-88b66d5ee8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:41.968296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74292d-f0cd-45ef-91c3-88b66d5ee8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:41.988210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57dd68408eb14eda9586e4d1283a563e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:41.988918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57dd68408eb14eda9586e4d1283a563e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:41.990243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57dd68408eb14eda9586e4d1283a563e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:41.993096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74292d-f0cd-45ef-91c3-88b66d5ee8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:41.997209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74292d-f0cd-45ef-91c3-88b66d5ee8f9 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:42.008804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57dd68408eb14eda9586e4d1283a563e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:42.015637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 849/1695] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:42.020094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1cdc-b914-4d24-a092-d0133b8d3974 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:42.022326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1cdc-b914-4d24-a092-d0133b8d3974 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:42.022825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1cdc-b914-4d24-a092-d0133b8d3974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:42.026900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99e357d9-4db3-4e1f-83f1-baa1b579f094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57dd68408eb14eda9586e4d1283a563e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:42.062374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-105a1cdc-b914-4d24-a092-d0133b8d3974 None None] MFA Rules not processed for user `8ab2da92b7e94592b80ef3d7852f51ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:42.065551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-105a1cdc-b914-4d24-a092-d0133b8d3974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:42.077624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 850/1696] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:42.078781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a1cdc-b914-4d24-a092-d0133b8d3974 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:42.081539 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-105a1cdc-b914-4d24-a092-d0133b8d3974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:42.129912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 847/1697] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:41 2022] DELETE /identity/v3/users/57dd68408eb14eda9586e4d1283a563e => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:42.129912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515d95-9af0-402a-aa61-8e20672ddef7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:42.129912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515d95-9af0-402a-aa61-8e20672ddef7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:42.129912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515d95-9af0-402a-aa61-8e20672ddef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:42.151639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a1cdc-b914-4d24-a092-d0133b8d3974 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:42.193374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2e515d95-9af0-402a-aa61-8e20672ddef7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:42.195911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2e515d95-9af0-402a-aa61-8e20672ddef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:42.341233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f701b0-a86a-4cb6-b3b2-80186bd85a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:42.342533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f701b0-a86a-4cb6-b3b2-80186bd85a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:42.343091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f701b0-a86a-4cb6-b3b2-80186bd85a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:42.343613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f701b0-a86a-4cb6-b3b2-80186bd85a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:42.344577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f701b0-a86a-4cb6-b3b2-80186bd85a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:42.350489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f701b0-a86a-4cb6-b3b2-80186bd85a5c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:42.373916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 851/1698] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:42.376603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e717-46ae-461d-8112-f8b485b8a6ee None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:42.377107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e717-46ae-461d-8112-f8b485b8a6ee None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:42.377740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e717-46ae-461d-8112-f8b485b8a6ee None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:42.380796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 852/1699] 10.222.0.20 () {54 vars in 912 bytes} [Fri Jul 1 04:04:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:04:42.382374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e717-46ae-461d-8112-f8b485b8a6ee None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:42.388324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5701e717-46ae-461d-8112-f8b485b8a6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:42.431824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 848/1700] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:42.433283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f214-f25d-4398-b3af-bb2526ac4c93 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:42.433560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f214-f25d-4398-b3af-bb2526ac4c93 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:42.436594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f214-f25d-4398-b3af-bb2526ac4c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:42.484486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-17a5f214-f25d-4398-b3af-bb2526ac4c93 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:42.485810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-17a5f214-f25d-4398-b3af-bb2526ac4c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:42.510398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 849/1701] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:42.511801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5f214-f25d-4398-b3af-bb2526ac4c93 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:42.534039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5f214-f25d-4398-b3af-bb2526ac4c93 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:42.542882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92d2bb-e2a5-450e-a780-adb7bc5ad685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:42.544203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d2bb-e2a5-450e-a780-adb7bc5ad685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:42.544735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d2bb-e2a5-450e-a780-adb7bc5ad685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:42.545232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d2bb-e2a5-450e-a780-adb7bc5ad685 admin admin] PATH_INFO: `/v3/projects/e0e98b34543f4717bbd76c6000db1b60/users/e9e58c55e1524f5f88e645f664741a19/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:42.584297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92d2bb-e2a5-450e-a780-adb7bc5ad685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e98b34543f4717bbd76c6000db1b60, user_id=e9e58c55e1524f5f88e645f664741a19, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:42.587309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92d2bb-e2a5-450e-a780-adb7bc5ad685 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:42.591259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e717-46ae-461d-8112-f8b485b8a6ee None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:42.606175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:42.607181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:42.607606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:42.608208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:42.612663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:42.751056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 850/1702] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:42 2022] PUT /identity/v3/projects/e0e98b34543f4717bbd76c6000db1b60/users/e9e58c55e1524f5f88e645f664741a19/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:42.754335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc749-1bb4-4404-a34f-ae1bffa621a6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:42.754773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc749-1bb4-4404-a34f-ae1bffa621a6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:42.755520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc749-1bb4-4404-a34f-ae1bffa621a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:42.789594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:42.794684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:42.832387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 853/1703] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:42.834204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cebcc749-1bb4-4404-a34f-ae1bffa621a6 None None] MFA Rules not processed for user `0beff646206e446386d91537df9eaa67`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:42.836200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cebcc749-1bb4-4404-a34f-ae1bffa621a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:42.843607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:42.860890 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8a886-1cd0-40ac-b438-54a2d88fc23d service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:42.985603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 851/1704] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:04:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:42.992897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcc749-1bb4-4404-a34f-ae1bffa621a6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:42.992897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cebcc749-1bb4-4404-a34f-ae1bffa621a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:43.019180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcc749-1bb4-4404-a34f-ae1bffa621a6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:43.028940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a68034-8bc0-437a-8b5d-89b3fa8a25fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:43.031711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a68034-8bc0-437a-8b5d-89b3fa8a25fb service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.031996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a68034-8bc0-437a-8b5d-89b3fa8a25fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.032267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a68034-8bc0-437a-8b5d-89b3fa8a25fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.076633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3160656-6faa-4abe-aeb5-08dddc9f27fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:43.076633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3160656-6faa-4abe-aeb5-08dddc9f27fe admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.076633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3160656-6faa-4abe-aeb5-08dddc9f27fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.076633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3160656-6faa-4abe-aeb5-08dddc9f27fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.077113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3160656-6faa-4abe-aeb5-08dddc9f27fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:43.077113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3160656-6faa-4abe-aeb5-08dddc9f27fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:43.078199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3160656-6faa-4abe-aeb5-08dddc9f27fe admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:43.135752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 854/1705] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:43.135752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b8c72-75d4-4a01-bb2e-9935e7ba7d66 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.135752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b8c72-75d4-4a01-bb2e-9935e7ba7d66 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.135752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b8c72-75d4-4a01-bb2e-9935e7ba7d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.175552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:43.197913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b43b8c72-75d4-4a01-bb2e-9935e7ba7d66 None None] MFA Rules not processed for user `708734ba3ddc49a3a9dba1d891aeccc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:43.199407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b43b8c72-75d4-4a01-bb2e-9935e7ba7d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:43.225003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a68034-8bc0-437a-8b5d-89b3fa8a25fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:43.227154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a68034-8bc0-437a-8b5d-89b3fa8a25fb service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:43.267887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 852/1706] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:43.269229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a68034-8bc0-437a-8b5d-89b3fa8a25fb service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:43.302125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a68034-8bc0-437a-8b5d-89b3fa8a25fb service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:43.321252 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82e1c2-55be-4684-9f72-69e8b47e6d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:43.322676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 855/1707] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:43.323779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b8c72-75d4-4a01-bb2e-9935e7ba7d66 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:43.324856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82e1c2-55be-4684-9f72-69e8b47e6d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.325494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82e1c2-55be-4684-9f72-69e8b47e6d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.325866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b43b8c72-75d4-4a01-bb2e-9935e7ba7d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:43.327975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82e1c2-55be-4684-9f72-69e8b47e6d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.330803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82e1c2-55be-4684-9f72-69e8b47e6d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:43.338954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82e1c2-55be-4684-9f72-69e8b47e6d02 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:43.342181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b8c72-75d4-4a01-bb2e-9935e7ba7d66 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:43.348446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:43.349765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.350072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.350411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] PATH_INFO: `/v3/projects/7e313ea341f54449b01ac438c507885c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.351848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e313ea341f54449b01ac438c507885c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:43.354107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:43.391645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 853/1708] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:43.392366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82e1c2-55be-4684-9f72-69e8b47e6d02 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:43.421611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82e1c2-55be-4684-9f72-69e8b47e6d02 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:43.441546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccf075-6a92-4851-aab0-37c3c150c14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:43.442246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf075-6a92-4851-aab0-37c3c150c14c service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.442528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf075-6a92-4851-aab0-37c3c150c14c service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.442804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf075-6a92-4851-aab0-37c3c150c14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.444268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:43.452129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccf075-6a92-4851-aab0-37c3c150c14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:43.453660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccf075-6a92-4851-aab0-37c3c150c14c service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:43.464367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 854/1709] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:43.465295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccf075-6a92-4851-aab0-37c3c150c14c service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:43.480925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccf075-6a92-4851-aab0-37c3c150c14c service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:43.486832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ce68b-5b0a-483c-8224-1053367ec300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:43.487605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce68b-5b0a-483c-8224-1053367ec300 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.487887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce68b-5b0a-483c-8224-1053367ec300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.488212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce68b-5b0a-483c-8224-1053367ec300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.489334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ce68b-5b0a-483c-8224-1053367ec300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:43.491665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ce68b-5b0a-483c-8224-1053367ec300 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:43.556503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e313ea341f54449b01ac438c507885c', 'request_id': 'req-0cb69c2c-7232-4743-bcaa-208e2df32b84'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:43.580561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e313ea341f54449b01ac438c507885c', 'request_id': 'req-0cb69c2c-7232-4743-bcaa-208e2df32b84'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:43.598293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e313ea341f54449b01ac438c507885c', 'request_id': 'req-0cb69c2c-7232-4743-bcaa-208e2df32b84'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:43.712102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 855/1710] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:43 2022] POST /identity/v3/projects => generated 355 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:43.713044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ce68b-5b0a-483c-8224-1053367ec300 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:43.743257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] The token cache is being invalidate because project 7e313ea341f54449b01ac438c507885c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:43.743496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:43.750502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0cb69c2c-7232-4743-bcaa-208e2df32b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:43.750502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 856/1711] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:43 2022] DELETE /identity/v3/projects/7e313ea341f54449b01ac438c507885c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:43.750502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c100-3ee5-4086-8ff6-1aa7163b62ac None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.750502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c100-3ee5-4086-8ff6-1aa7163b62ac None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.750502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c100-3ee5-4086-8ff6-1aa7163b62ac None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.754134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 857/1712] 10.222.0.20 () {54 vars in 912 bytes} [Fri Jul 1 04:04:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:04:43.755039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7c100-3ee5-4086-8ff6-1aa7163b62ac None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:43.757190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-02e7c100-3ee5-4086-8ff6-1aa7163b62ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:43.881723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:43.934990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7c100-3ee5-4086-8ff6-1aa7163b62ac None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:43.949963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bb21c-512c-47e7-94b9-30465ec86784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:43.950636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ce68b-5b0a-483c-8224-1053367ec300 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:43.951100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bb21c-512c-47e7-94b9-30465ec86784 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.951482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bb21c-512c-47e7-94b9-30465ec86784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.951827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bb21c-512c-47e7-94b9-30465ec86784 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.952701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bb21c-512c-47e7-94b9-30465ec86784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:43.959592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bb21c-512c-47e7-94b9-30465ec86784 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:43.967849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:43.969476 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:43.971275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:43.971275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] PATH_INFO: `/v3/projects/e0e98b34543f4717bbd76c6000db1b60/users/e9e58c55e1524f5f88e645f664741a19/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:43.977016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e98b34543f4717bbd76c6000db1b60, user_id=e9e58c55e1524f5f88e645f664741a19, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:43.981697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:44.005484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 856/1713] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:43 2022] PUT /identity/v3/projects/e0e98b34543f4717bbd76c6000db1b60/users/e9e58c55e1524f5f88e645f664741a19/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:44.006706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:44.008876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:44.088252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 858/1714] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:43 2022] POST /identity/v3/users => generated 278 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:44.089795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e2de5-bdf3-40b5-920a-3467124d1469 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:44.090091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e2de5-bdf3-40b5-920a-3467124d1469 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:44.090369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e2de5-bdf3-40b5-920a-3467124d1469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:44.177244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ee5e2de5-bdf3-40b5-920a-3467124d1469 None None] MFA Rules not processed for user `0beff646206e446386d91537df9eaa67`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:44.178653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ee5e2de5-bdf3-40b5-920a-3467124d1469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:44.327445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 859/1715] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:04:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:44.330555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2b16-e12f-4308-8605-c1bb3fb89a11 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:44.333084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2b16-e12f-4308-8605-c1bb3fb89a11 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:44.333705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2b16-e12f-4308-8605-c1bb3fb89a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:44.334204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc4280-5d80-4db6-b4ba-6b8fbbfd440e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:44.346249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:44.360118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:44.360118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:44.360118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] PATH_INFO: `/v3/users/2d13e03a9b7c442391b9f4600966c4e4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:44.360118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d13e03a9b7c442391b9f4600966c4e4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:44.360118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:44.381729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:44.435009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8c3c2b16-e12f-4308-8605-c1bb3fb89a11 None None] MFA Rules not processed for user `e9e58c55e1524f5f88e645f664741a19`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:44.437997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8c3c2b16-e12f-4308-8605-c1bb3fb89a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:44.670994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 860/1716] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:44.670994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2b16-e12f-4308-8605-c1bb3fb89a11 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:44.701842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2b16-e12f-4308-8605-c1bb3fb89a11 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:44.734048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:44.734932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:44.735569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:44.735962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:44.737122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:44.739193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:44.761667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 861/1717] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:44.762721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:44.769029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:44.794324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d13e03a9b7c442391b9f4600966c4e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:44.795938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d13e03a9b7c442391b9f4600966c4e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:44.797902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d13e03a9b7c442391b9f4600966c4e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:44.806774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932b7b8-5032-41a0-a68f-71377e07f707 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:44.821452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:44.822361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:44.822643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:44.822919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] PATH_INFO: `/v3/projects/f80f7c3079364d238d394945f393bb75` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:44.832267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80f7c3079364d238d394945f393bb75)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:44.834524 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:44.866169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d13e03a9b7c442391b9f4600966c4e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:44.900658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d13e03a9b7c442391b9f4600966c4e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:45.047815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 857/1718] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:44 2022] DELETE /identity/v3/users/2d13e03a9b7c442391b9f4600966c4e4 => generated 0 bytes in 1038 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:45.049334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:45.052798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:45.117586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80f7c3079364d238d394945f393bb75', 'request_id': 'req-295e3a94-3ba2-4178-8b67-a8dad4fef89c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:45.132316 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80f7c3079364d238d394945f393bb75', 'request_id': 'req-295e3a94-3ba2-4178-8b67-a8dad4fef89c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:45.150123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80f7c3079364d238d394945f393bb75', 'request_id': 'req-295e3a94-3ba2-4178-8b67-a8dad4fef89c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:45.222263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fd671-7653-4c0d-98a9-0b646327c3f8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:45.305634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] The token cache is being invalidate because project f80f7c3079364d238d394945f393bb75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:45.305937 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:45.307833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:45.313266 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 862/1719] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:44 2022] DELETE /identity/v3/projects/f80f7c3079364d238d394945f393bb75 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:45.314290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:45.320081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:45.353551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7fdbb-dfe3-44a1-a80c-442716c268e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:45.354959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7fdbb-dfe3-44a1-a80c-442716c268e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:45.356732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7fdbb-dfe3-44a1-a80c-442716c268e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:45.357389 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7fdbb-dfe3-44a1-a80c-442716c268e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:45.388509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-88d7fdbb-dfe3-44a1-a80c-442716c268e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:45.443533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7fdbb-dfe3-44a1-a80c-442716c268e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:45.451953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7fdbb-dfe3-44a1-a80c-442716c268e9 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:45.503707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e3a94-3ba2-4178-8b67-a8dad4fef89c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:45.512390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 858/1720] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:45.515903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ba393-b659-4b5c-b72a-30c08fbd9874 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:45.517766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ba393-b659-4b5c-b72a-30c08fbd9874 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:45.518684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ba393-b659-4b5c-b72a-30c08fbd9874 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:45.522462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 859/1721] 10.222.0.20 () {54 vars in 960 bytes} [Fri Jul 1 04:04:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:04:45.524233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ba393-b659-4b5c-b72a-30c08fbd9874 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:45.529387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d42ba393-b659-4b5c-b72a-30c08fbd9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:45.558082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:45.558669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:45.558958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:45.559269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:45.565094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:45.624523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:45.717951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:45.725313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:45.801019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 863/1722] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:45.804959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:45.807054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ba393-b659-4b5c-b72a-30c08fbd9874 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:45.813012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:45.822376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4a733-e065-41b6-aa9c-b7eab2f671af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:45.828277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4a733-e065-41b6-aa9c-b7eab2f671af service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:45.828870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4a733-e065-41b6-aa9c-b7eab2f671af service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:45.830342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4a733-e065-41b6-aa9c-b7eab2f671af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:45.844589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-20a4a733-e065-41b6-aa9c-b7eab2f671af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:46.192338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4a733-e065-41b6-aa9c-b7eab2f671af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:46.201063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4a733-e065-41b6-aa9c-b7eab2f671af service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:46.238137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 860/1723] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:46.241107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3058c-bc32-4d6b-96ad-3b776d838fc0 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:46.243098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ebacc-fb43-4ac7-a9e2-79165cf77d8e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:46.244254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ebacc-fb43-4ac7-a9e2-79165cf77d8e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:46.245783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ebacc-fb43-4ac7-a9e2-79165cf77d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:46.287792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:46.302450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:46.302450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:46.302450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:46.302450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:46.304959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:46.315436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d27ebacc-fb43-4ac7-a9e2-79165cf77d8e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:46.323000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d27ebacc-fb43-4ac7-a9e2-79165cf77d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:46.381253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 864/1724] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 555 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:46.382763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 861/1725] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:46.385136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e643e50-aa34-4aff-80fd-4a830445be16 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:46.385428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e643e50-aa34-4aff-80fd-4a830445be16 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:46.385711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e643e50-aa34-4aff-80fd-4a830445be16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:46.393263 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:46.400797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:46.442493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:46.500606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563f1a6-b9b4-48d0-8439-42a4f22a717e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:46.510013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6e643e50-aa34-4aff-80fd-4a830445be16 None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:46.514411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6e643e50-aa34-4aff-80fd-4a830445be16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:46.522178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:46.523671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:46.523964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:46.526452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] PATH_INFO: `/v3/users/afed68a2c1db4507b71ce0c850550937` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:46.529002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afed68a2c1db4507b71ce0c850550937)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:46.536372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:46.854332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 862/1726] 10.222.0.20 () {58 vars in 1057 bytes} [Fri Jul 1 04:04:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:46.865955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e643e50-aa34-4aff-80fd-4a830445be16 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:46.880793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6e643e50-aa34-4aff-80fd-4a830445be16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:46.913164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afed68a2c1db4507b71ce0c850550937'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:46.916523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afed68a2c1db4507b71ce0c850550937'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:46.992256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e643e50-aa34-4aff-80fd-4a830445be16 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:47.007051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:47.007993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:47.008281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:47.008557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] PATH_INFO: `/v3/projects/b0024ecc55604d69be1d2a5dca5af9cd` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:47.010291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0024ecc55604d69be1d2a5dca5af9cd)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:47.012425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:47.132192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afed68a2c1db4507b71ce0c850550937'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:47.152226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afed68a2c1db4507b71ce0c850550937'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:47.153469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afed68a2c1db4507b71ce0c850550937'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:47.243696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 865/1727] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:46 2022] DELETE /identity/v3/users/afed68a2c1db4507b71ce0c850550937 => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:47.245391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:47.249639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:47.323562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0024ecc55604d69be1d2a5dca5af9cd', 'request_id': 'req-97954fa7-2c0d-4265-92ed-411424b44798'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:47.334820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0024ecc55604d69be1d2a5dca5af9cd', 'request_id': 'req-97954fa7-2c0d-4265-92ed-411424b44798'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:47.396420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4ac84-7053-42b5-8fab-804dd7abda52 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:47.505835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0024ecc55604d69be1d2a5dca5af9cd', 'request_id': 'req-97954fa7-2c0d-4265-92ed-411424b44798'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:47.529736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] The token cache is being invalidate because project b0024ecc55604d69be1d2a5dca5af9cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:47.530032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:47.530775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-97954fa7-2c0d-4265-92ed-411424b44798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:47.536749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 863/1728] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:46 2022] DELETE /identity/v3/projects/b0024ecc55604d69be1d2a5dca5af9cd => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:47.537419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01212a-0c18-4169-b62f-0bee60ecf00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:47.538420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01212a-0c18-4169-b62f-0bee60ecf00e admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:47.538708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01212a-0c18-4169-b62f-0bee60ecf00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:47.538993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01212a-0c18-4169-b62f-0bee60ecf00e admin admin] PATH_INFO: `/v3/projects/432d6d42bc994329834ec0de75dade1b/users/2c4dab23ea9a44bfb8db3d5d4abb8017/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:47.539703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c57a33-bfd6-45d2-bc5a-6f90485119a3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:47.540144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c57a33-bfd6-45d2-bc5a-6f90485119a3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:47.540581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c57a33-bfd6-45d2-bc5a-6f90485119a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:47.584443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b01212a-0c18-4169-b62f-0bee60ecf00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=432d6d42bc994329834ec0de75dade1b, user_id=2c4dab23ea9a44bfb8db3d5d4abb8017, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:47.587827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a3c57a33-bfd6-45d2-bc5a-6f90485119a3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:47.589451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a3c57a33-bfd6-45d2-bc5a-6f90485119a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:47.591932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b01212a-0c18-4169-b62f-0bee60ecf00e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:47.614177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 864/1729] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:47.616096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd29a4-06be-421f-ae13-2cb7c510ad69 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:47.616449 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd29a4-06be-421f-ae13-2cb7c510ad69 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:47.616883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd29a4-06be-421f-ae13-2cb7c510ad69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:47.650878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1cbd29a4-06be-421f-ae13-2cb7c510ad69 None None] MFA Rules not processed for user `8622ec5212c84d87b9b7c8f725012a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:47.653116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1cbd29a4-06be-421f-ae13-2cb7c510ad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:47.739012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 865/1730] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:47.739959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd29a4-06be-421f-ae13-2cb7c510ad69 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:47.748249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1cbd29a4-06be-421f-ae13-2cb7c510ad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:47.776321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 866/1731] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:47 2022] PUT /identity/v3/projects/432d6d42bc994329834ec0de75dade1b/users/2c4dab23ea9a44bfb8db3d5d4abb8017/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:47.779714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524bc4-f08b-4e2a-9a06-adb82d428957 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:47.784896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524bc4-f08b-4e2a-9a06-adb82d428957 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:47.784896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524bc4-f08b-4e2a-9a06-adb82d428957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:47.855502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1f524bc4-f08b-4e2a-9a06-adb82d428957 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:47.858734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1f524bc4-f08b-4e2a-9a06-adb82d428957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:47.958671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd29a4-06be-421f-ae13-2cb7c510ad69 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:48.085850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 867/1732] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:48.088002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524bc4-f08b-4e2a-9a06-adb82d428957 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:48.091100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1f524bc4-f08b-4e2a-9a06-adb82d428957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:48.112572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94949d-10f5-4af4-a455-26f6cd75fe37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:48.114075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94949d-10f5-4af4-a455-26f6cd75fe37 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.114659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94949d-10f5-4af4-a455-26f6cd75fe37 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.115268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94949d-10f5-4af4-a455-26f6cd75fe37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.118832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1a94949d-10f5-4af4-a455-26f6cd75fe37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:48.164324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94949d-10f5-4af4-a455-26f6cd75fe37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:48.168844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94949d-10f5-4af4-a455-26f6cd75fe37 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:48.206888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 866/1733] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:48.210311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524bc4-f08b-4e2a-9a06-adb82d428957 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:48.210563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52ff1f-39ea-404c-a515-64ff67e3b42a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.211973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52ff1f-39ea-404c-a515-64ff67e3b42a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.212273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52ff1f-39ea-404c-a515-64ff67e3b42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.229944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abce1d-d577-4a1f-acf0-f48b3bf77e50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:48.230645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abce1d-d577-4a1f-acf0-f48b3bf77e50 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.230935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abce1d-d577-4a1f-acf0-f48b3bf77e50 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.231210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abce1d-d577-4a1f-acf0-f48b3bf77e50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.254425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abce1d-d577-4a1f-acf0-f48b3bf77e50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:48.256467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abce1d-d577-4a1f-acf0-f48b3bf77e50 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:48.268473 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5f52ff1f-39ea-404c-a515-64ff67e3b42a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:48.272735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5f52ff1f-39ea-404c-a515-64ff67e3b42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:48.279624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 868/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:48.280580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abce1d-d577-4a1f-acf0-f48b3bf77e50 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:48.299797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abce1d-d577-4a1f-acf0-f48b3bf77e50 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:48.328615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 867/1735] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:48.332198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d1dd-82d5-41f1-b2a5-115e5dcd28f7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.339289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d1dd-82d5-41f1-b2a5-115e5dcd28f7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.339852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d1dd-82d5-41f1-b2a5-115e5dcd28f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.347281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:48.348785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.350942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:48.355215 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.355708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.356517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:48.356838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:48.358514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:48.420509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 869/1736] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:48.420862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:48.425829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:48.467608 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3a60d1dd-82d5-41f1-b2a5-115e5dcd28f7 None None] MFA Rules not processed for user `2c4dab23ea9a44bfb8db3d5d4abb8017`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:48.469399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3a60d1dd-82d5-41f1-b2a5-115e5dcd28f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:48.590693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:48.662235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420cc59-58d0-45b4-b317-5efb1f0b24f2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:48.671951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab69c9-1a09-4358-8d39-dc1195016fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:48.672704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab69c9-1a09-4358-8d39-dc1195016fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.673007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab69c9-1a09-4358-8d39-dc1195016fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.673288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab69c9-1a09-4358-8d39-dc1195016fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.699878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab69c9-1a09-4358-8d39-dc1195016fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:48.701356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab69c9-1a09-4358-8d39-dc1195016fe5 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:48.716363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 868/1737] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:48 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:48.720589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60d1dd-82d5-41f1-b2a5-115e5dcd28f7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:48.726630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3a60d1dd-82d5-41f1-b2a5-115e5dcd28f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:48.744512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 870/1738] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:48.751739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5599d6-e396-4839-9319-198f28d32bf4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.752260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5599d6-e396-4839-9319-198f28d32bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.752686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5599d6-e396-4839-9319-198f28d32bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.798258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-cc5599d6-e396-4839-9319-198f28d32bf4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:48.799573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cc5599d6-e396-4839-9319-198f28d32bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:48.800815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60d1dd-82d5-41f1-b2a5-115e5dcd28f7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:48.819681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 871/1739] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:48.820610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5599d6-e396-4839-9319-198f28d32bf4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:48.822837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:48.825200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.826019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.833918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5599d6-e396-4839-9319-198f28d32bf4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:48.836630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] PATH_INFO: `/v3/users/7947bb65ecad43e89376ce6afa100d22` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.837807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7947bb65ecad43e89376ce6afa100d22)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:48.848648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:48.849081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:48.849336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:48.849624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:48.849906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:48.850618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:48.852777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:49.175136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 872/1740] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:48 2022] POST /identity/v3/projects => generated 371 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:49.176856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:49.180592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:49.332245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:49.406110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7947bb65ecad43e89376ce6afa100d22'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:49.411742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7947bb65ecad43e89376ce6afa100d22'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:49.421152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7947bb65ecad43e89376ce6afa100d22'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:49.460332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b8c14-d59b-469c-b928-310d19286a1a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:49.470915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:49.472389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:49.473046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:49.473806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:49.475107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:49.479880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:49.501902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7947bb65ecad43e89376ce6afa100d22'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:49.546922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7947bb65ecad43e89376ce6afa100d22'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:49.742489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 869/1741] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:48 2022] DELETE /identity/v3/users/7947bb65ecad43e89376ce6afa100d22 => generated 0 bytes in 1023 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:49.744270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:49.746710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:49.775969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 873/1742] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:49 2022] POST /identity/v3/projects => generated 355 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:49.780002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:49.791477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:50.030599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef741-4c0a-4388-b6db-776d3ee6521d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:50.079059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:50.244187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1044d-3079-42ef-ad1e-260a6ee78433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:50.244778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1044d-3079-42ef-ad1e-260a6ee78433 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:50.245063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1044d-3079-42ef-ad1e-260a6ee78433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:50.245343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1044d-3079-42ef-ad1e-260a6ee78433 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:50.246095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1044d-3079-42ef-ad1e-260a6ee78433 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:50.246961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1044d-3079-42ef-ad1e-260a6ee78433 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:50.248262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1044d-3079-42ef-ad1e-260a6ee78433 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:50.294783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 870/1743] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 551 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:50.296125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dfa6b-bda3-4717-9552-c1f81572f27f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:50.296486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dfa6b-bda3-4717-9552-c1f81572f27f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:50.296695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dfa6b-bda3-4717-9552-c1f81572f27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:50.357851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-719dfa6b-bda3-4717-9552-c1f81572f27f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:50.360159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-719dfa6b-bda3-4717-9552-c1f81572f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:50.370730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4537a2-d7a7-4afe-8815-b668205aaea2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:50.376899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 871/1744] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:50.378264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dfa6b-bda3-4717-9552-c1f81572f27f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:50.382666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-719dfa6b-bda3-4717-9552-c1f81572f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:50.396360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:50.400589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:50.401144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:50.401652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:50.412380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:50.417285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:50.429573 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dfa6b-bda3-4717-9552-c1f81572f27f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:50.441396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093ac62-ca13-41ad-ab0f-c359edfed447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:50.442566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093ac62-ca13-41ad-ab0f-c359edfed447 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:50.443138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093ac62-ca13-41ad-ab0f-c359edfed447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:50.452517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093ac62-ca13-41ad-ab0f-c359edfed447 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:50.452517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093ac62-ca13-41ad-ab0f-c359edfed447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:50.452517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093ac62-ca13-41ad-ab0f-c359edfed447 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:50.650855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 872/1745] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:50 2022] POST /identity/v3/users => generated 284 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:50.652449 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093ac62-ca13-41ad-ab0f-c359edfed447 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:50.654740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 874/1746] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:49 2022] POST /identity/v3/users => generated 286 bytes in 881 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:50.658677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:50.670635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:50.725633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093ac62-ca13-41ad-ab0f-c359edfed447 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:50.750616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45580de7-e3a3-4466-8c4f-7bde151a1e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:50.752507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45580de7-e3a3-4466-8c4f-7bde151a1e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:50.753100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45580de7-e3a3-4466-8c4f-7bde151a1e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:50.753801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45580de7-e3a3-4466-8c4f-7bde151a1e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:50.755161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45580de7-e3a3-4466-8c4f-7bde151a1e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:50.765327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45580de7-e3a3-4466-8c4f-7bde151a1e37 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:50.966532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 873/1747] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:50 2022] POST /identity/v3/projects => generated 381 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:50.966532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45580de7-e3a3-4466-8c4f-7bde151a1e37 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:51.088055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905898b5-d554-4461-aa0c-1c911d3f7990 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:51.277353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45580de7-e3a3-4466-8c4f-7bde151a1e37 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:51.292054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128c127-0b57-4e32-8125-ecee413068fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:51.293236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128c127-0b57-4e32-8125-ecee413068fd admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:51.293598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128c127-0b57-4e32-8125-ecee413068fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:51.294565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128c127-0b57-4e32-8125-ecee413068fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:51.296742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128c127-0b57-4e32-8125-ecee413068fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:51.303354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128c127-0b57-4e32-8125-ecee413068fd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:51.306074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:51.306925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:51.307244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:51.307550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:51.333648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 874/1748] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:51.336654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128c127-0b57-4e32-8125-ecee413068fd admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:51.360835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:51.366272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:51.383801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128c127-0b57-4e32-8125-ecee413068fd admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:51.398576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 875/1749] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 741 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:51.399540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:51.401348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:51.415056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d1e31-f6fb-4d7b-9489-71eef0544dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:51.415667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1e31-f6fb-4d7b-9489-71eef0544dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:51.415994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1e31-f6fb-4d7b-9489-71eef0544dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:51.420518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1e31-f6fb-4d7b-9489-71eef0544dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:51.420518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1e31-f6fb-4d7b-9489-71eef0544dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:51.420518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1e31-f6fb-4d7b-9489-71eef0544dd0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:51.439952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481361f7-bbe8-4895-977d-3ca91791112d service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:51.444343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 875/1750] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:51.445433 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d1e31-f6fb-4d7b-9489-71eef0544dd0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:51.454809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:51.457845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:51.458955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:51.461392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] PATH_INFO: `/v3/projects/dca239d889d64af38ae3ba99fd9b41e7` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:51.461477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d1e31-f6fb-4d7b-9489-71eef0544dd0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:51.463487 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca239d889d64af38ae3ba99fd9b41e7)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:51.470939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:51.483381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182f19e-a9a5-47fd-b9cf-dc4e2c08eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:51.484243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182f19e-a9a5-47fd-b9cf-dc4e2c08eab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:51.484567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182f19e-a9a5-47fd-b9cf-dc4e2c08eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:51.484869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182f19e-a9a5-47fd-b9cf-dc4e2c08eab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:51.485892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182f19e-a9a5-47fd-b9cf-dc4e2c08eab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:51.500708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182f19e-a9a5-47fd-b9cf-dc4e2c08eab6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:51.628190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 876/1751] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:51 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:51.629213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182f19e-a9a5-47fd-b9cf-dc4e2c08eab6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:51.656674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182f19e-a9a5-47fd-b9cf-dc4e2c08eab6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:51.680084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19acb7fb-c5e6-4299-acbe-1970fcd8e881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:51.681849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acb7fb-c5e6-4299-acbe-1970fcd8e881 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:51.682740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acb7fb-c5e6-4299-acbe-1970fcd8e881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:51.683516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acb7fb-c5e6-4299-acbe-1970fcd8e881 admin admin] PATH_INFO: `/v3/projects/74b568677fe44467b3b6bdefeb8c142a/users/7ae36b3a04814e5a97ba7ba743b780d9/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:51.763603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19acb7fb-c5e6-4299-acbe-1970fcd8e881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b568677fe44467b3b6bdefeb8c142a, user_id=7ae36b3a04814e5a97ba7ba743b780d9, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:51.772438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19acb7fb-c5e6-4299-acbe-1970fcd8e881 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:51.807395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca239d889d64af38ae3ba99fd9b41e7', 'request_id': 'req-b8db8cb1-9201-4e5b-a866-c171be3846d6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:51.835575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca239d889d64af38ae3ba99fd9b41e7', 'request_id': 'req-b8db8cb1-9201-4e5b-a866-c171be3846d6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:51.851789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca239d889d64af38ae3ba99fd9b41e7', 'request_id': 'req-b8db8cb1-9201-4e5b-a866-c171be3846d6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:51.924714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 877/1752] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:51 2022] PUT /identity/v3/projects/74b568677fe44467b3b6bdefeb8c142a/users/7ae36b3a04814e5a97ba7ba743b780d9/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:51.925985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19acb7fb-c5e6-4299-acbe-1970fcd8e881 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:51.955982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] The token cache is being invalidate because project dca239d889d64af38ae3ba99fd9b41e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:51.962845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:51.962845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:51.962845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 876/1753] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:51 2022] DELETE /identity/v3/projects/dca239d889d64af38ae3ba99fd9b41e7 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:51.962845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:51.966076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:52.101722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19acb7fb-c5e6-4299-acbe-1970fcd8e881 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:52.108939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:52.120036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.124655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.124970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.127994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:52.129378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8cb1-9201-4e5b-a866-c171be3846d6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:52.133789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:52.135819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:52.149411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b01b64-7d68-46e3-bdd4-6d61c46ab99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:52.153360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01b64-7d68-46e3-bdd4-6d61c46ab99b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.153989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01b64-7d68-46e3-bdd4-6d61c46ab99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.154708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01b64-7d68-46e3-bdd4-6d61c46ab99b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.156428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01b64-7d68-46e3-bdd4-6d61c46ab99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:52.161852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01b64-7d68-46e3-bdd4-6d61c46ab99b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:52.167321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 878/1754] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:52.168246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:52.170144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:52.189259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f7ced-b3c2-4649-8a6f-4b14802af64c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:52.197586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bff18-684c-48fa-af9a-3b0322555579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:52.198599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bff18-684c-48fa-af9a-3b0322555579 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.198717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 877/1755] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:52.201195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e6ad5-7084-4764-9252-67d8de4230ab None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.201859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e6ad5-7084-4764-9252-67d8de4230ab None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.202449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e6ad5-7084-4764-9252-67d8de4230ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.215082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bff18-684c-48fa-af9a-3b0322555579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.215082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bff18-684c-48fa-af9a-3b0322555579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.215082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bff18-684c-48fa-af9a-3b0322555579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:52.217959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bff18-684c-48fa-af9a-3b0322555579 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:52.249331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 879/1756] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:52.251511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c84077-5cc7-4343-9b30-e4472f246eb0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.251872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c84077-5cc7-4343-9b30-e4472f246eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.252231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c84077-5cc7-4343-9b30-e4472f246eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.253073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-588e6ad5-7084-4764-9252-67d8de4230ab None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:52.255826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-588e6ad5-7084-4764-9252-67d8de4230ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:52.305776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a8c84077-5cc7-4343-9b30-e4472f246eb0 None None] MFA Rules not processed for user `708734ba3ddc49a3a9dba1d891aeccc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:52.307472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a8c84077-5cc7-4343-9b30-e4472f246eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:52.311993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 878/1757] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:52.314502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e6ad5-7084-4764-9252-67d8de4230ab None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:52.317456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-588e6ad5-7084-4764-9252-67d8de4230ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:52.368259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e6ad5-7084-4764-9252-67d8de4230ab None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:52.383197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3ecb9-25ab-4348-a655-051008291b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:52.385171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3ecb9-25ab-4348-a655-051008291b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.385670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3ecb9-25ab-4348-a655-051008291b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.386158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3ecb9-25ab-4348-a655-051008291b43 admin admin] PATH_INFO: `/v3/projects/c79c8683f6614a9a9d883be0553d6b7e/users/9abd4bb809fa40689e8b2ad77b936a0f/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.427006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3ecb9-25ab-4348-a655-051008291b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79c8683f6614a9a9d883be0553d6b7e, user_id=9abd4bb809fa40689e8b2ad77b936a0f, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:52.442249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3ecb9-25ab-4348-a655-051008291b43 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:52.502740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 880/1758] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:52.508737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c84077-5cc7-4343-9b30-e4472f246eb0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:52.567148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c84077-5cc7-4343-9b30-e4472f246eb0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:52.586416 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f92160-5e8c-48b7-93b1-332d3275cb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:52.590378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f92160-5e8c-48b7-93b1-332d3275cb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.590668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f92160-5e8c-48b7-93b1-332d3275cb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.593186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f92160-5e8c-48b7-93b1-332d3275cb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.597462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f92160-5e8c-48b7-93b1-332d3275cb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:52.602672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f92160-5e8c-48b7-93b1-332d3275cb41 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:52.614105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:52.648166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 879/1759] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:52 2022] PUT /identity/v3/projects/c79c8683f6614a9a9d883be0553d6b7e/users/9abd4bb809fa40689e8b2ad77b936a0f/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:52.658457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3ecb9-25ab-4348-a655-051008291b43 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:52.667115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 881/1760] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:52.679955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e620c35-e8ab-4d15-9ef7-a439155c3d0e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.680248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e620c35-e8ab-4d15-9ef7-a439155c3d0e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.683169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e620c35-e8ab-4d15-9ef7-a439155c3d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.736192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4e620c35-e8ab-4d15-9ef7-a439155c3d0e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:52.737541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4e620c35-e8ab-4d15-9ef7-a439155c3d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:52.876185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3ecb9-25ab-4348-a655-051008291b43 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:52.888749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:52.889702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:52.889988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:52.896658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] PATH_INFO: `/v3/projects/74b568677fe44467b3b6bdefeb8c142a/users/7ae36b3a04814e5a97ba7ba743b780d9/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:52.907801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b568677fe44467b3b6bdefeb8c142a, user_id=7ae36b3a04814e5a97ba7ba743b780d9, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:52.913510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:52.931216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 880/1761] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:52 2022] PUT /identity/v3/projects/74b568677fe44467b3b6bdefeb8c142a/users/7ae36b3a04814e5a97ba7ba743b780d9/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:52.932315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:52.934074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:52.936355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 882/1762] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:52.939502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e620c35-e8ab-4d15-9ef7-a439155c3d0e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:52.989378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e620c35-e8ab-4d15-9ef7-a439155c3d0e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:53.124627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6bec8-6601-4148-abd6-756d9cdd8148 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:53.136976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:53.138470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:53.138847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:53.140572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] PATH_INFO: `/v3/projects/96cd59ca92d64483b4eafb41f68a2b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:53.145086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=96cd59ca92d64483b4eafb41f68a2b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:53.149927 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:53.164621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555424e3-afed-4349-b86a-dd5668a576a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:53.166168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555424e3-afed-4349-b86a-dd5668a576a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:53.167342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555424e3-afed-4349-b86a-dd5668a576a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:53.167827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555424e3-afed-4349-b86a-dd5668a576a4 admin admin] PATH_INFO: `/v3/projects/666e4d296dfd4500afdf68e39926b8d6/users/2bed8e3a24a6463d98fd793e72b19939/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:53.174824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:53.200793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555424e3-afed-4349-b86a-dd5668a576a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=666e4d296dfd4500afdf68e39926b8d6, user_id=2bed8e3a24a6463d98fd793e72b19939, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:53.210987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555424e3-afed-4349-b86a-dd5668a576a4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:53.368464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 883/1763] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:52 2022] PUT /identity/v3/projects/666e4d296dfd4500afdf68e39926b8d6/users/2bed8e3a24a6463d98fd793e72b19939/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:53.373119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45c102-0b00-491d-9717-91135c9a122a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:53.373856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45c102-0b00-491d-9717-91135c9a122a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:53.374136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45c102-0b00-491d-9717-91135c9a122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:53.395782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96cd59ca92d64483b4eafb41f68a2b9f', 'request_id': 'req-d04dd805-f80a-4c68-b2ca-efa753b8e69a'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:53.408059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96cd59ca92d64483b4eafb41f68a2b9f', 'request_id': 'req-d04dd805-f80a-4c68-b2ca-efa753b8e69a'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:53.425184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96cd59ca92d64483b4eafb41f68a2b9f', 'request_id': 'req-d04dd805-f80a-4c68-b2ca-efa753b8e69a'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:53.439755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ea45c102-0b00-491d-9717-91135c9a122a None None] MFA Rules not processed for user `9abd4bb809fa40689e8b2ad77b936a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:53.441016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ea45c102-0b00-491d-9717-91135c9a122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:53.564896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 884/1764] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:53.567946 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45c102-0b00-491d-9717-91135c9a122a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:53.587714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] The token cache is being invalidate because project 96cd59ca92d64483b4eafb41f68a2b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:53.588155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:53.589262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d04dd805-f80a-4c68-b2ca-efa753b8e69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:53.598365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 881/1765] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:52 2022] DELETE /identity/v3/projects/96cd59ca92d64483b4eafb41f68a2b9f => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:53.606642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fcd5b-9422-4ae4-9b53-ad5caecc188a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:53.606898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fcd5b-9422-4ae4-9b53-ad5caecc188a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:53.608552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fcd5b-9422-4ae4-9b53-ad5caecc188a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:53.669214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6d2fcd5b-9422-4ae4-9b53-ad5caecc188a None None] MFA Rules not processed for user `7ae36b3a04814e5a97ba7ba743b780d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:53.670788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6d2fcd5b-9422-4ae4-9b53-ad5caecc188a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:53.726778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45c102-0b00-491d-9717-91135c9a122a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:53.731907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8ab60-afd3-4a50-8fe2-d316926d066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:53.732646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8ab60-afd3-4a50-8fe2-d316926d066e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:53.732934 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8ab60-afd3-4a50-8fe2-d316926d066e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:53.733282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8ab60-afd3-4a50-8fe2-d316926d066e admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:53.734035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8ab60-afd3-4a50-8fe2-d316926d066e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:53.734527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8ab60-afd3-4a50-8fe2-d316926d066e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:53.736113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8ab60-afd3-4a50-8fe2-d316926d066e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:53.751562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 885/1766] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:53.753074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205642e-b5f6-4527-abab-2f78d5f0fe27 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:53.753371 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205642e-b5f6-4527-abab-2f78d5f0fe27 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:53.753650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205642e-b5f6-4527-abab-2f78d5f0fe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:53.781163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 882/1767] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:53 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:53.782944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fcd5b-9422-4ae4-9b53-ad5caecc188a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:53.786139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6d2fcd5b-9422-4ae4-9b53-ad5caecc188a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:53.788858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7205642e-b5f6-4527-abab-2f78d5f0fe27 None None] MFA Rules not processed for user `2bed8e3a24a6463d98fd793e72b19939`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:53.792766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7205642e-b5f6-4527-abab-2f78d5f0fe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:53.900363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:53.958840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fcd5b-9422-4ae4-9b53-ad5caecc188a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:53.967613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:53.969072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:53.969590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:53.970645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:53.975619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:53.980601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 886/1768] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:53.985014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205642e-b5f6-4527-abab-2f78d5f0fe27 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:53.988914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7205642e-b5f6-4527-abab-2f78d5f0fe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:53.989217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:53.990928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:54.002623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 883/1769] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:54.003567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:54.005180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:54.053045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205642e-b5f6-4527-abab-2f78d5f0fe27 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:54.060942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086dca5-7263-40da-9a0f-72ca84c2e098 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:54.078113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:54.078113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:54.078113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:54.078113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:54.080289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:54.080289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:54.083579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:54.085729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:54.085729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:54.086212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] PATH_INFO: `/v3/users/e9e58c55e1524f5f88e645f664741a19` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:54.089573 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9e58c55e1524f5f88e645f664741a19)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:54.103008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:54.354277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 884/1770] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:54 2022] POST /identity/v3/projects => generated 405 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:54.358973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:54.373677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:54.546718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9e58c55e1524f5f88e645f664741a19'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:54.548212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9e58c55e1524f5f88e645f664741a19'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:54.549774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9e58c55e1524f5f88e645f664741a19'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:54.568741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:54.575850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9e58c55e1524f5f88e645f664741a19'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:54.589629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9e58c55e1524f5f88e645f664741a19'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:54.618983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605d4f-fdd3-4dae-b0bc-d9b9693d9ee3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:54.626792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:54.628464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:54.629474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:54.630598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:54.634027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:54.775376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 887/1771] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:53 2022] DELETE /identity/v3/users/e9e58c55e1524f5f88e645f664741a19 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:54.775376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:54.780763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:54.844382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:54.847941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:54.872910 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 885/1772] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:54 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:54.875151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:54.878923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:55.049650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945ffb-458e-408e-9ad8-bcc7fcef95d3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:55.057625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:55.058603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:55.059033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:55.061357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:55.063133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:55.066239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:55.203729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:55.212916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03cb25-3f81-4f92-838c-b9fd2e8a1383 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:55.255036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768cdb5-b2e2-4228-ba0c-be268fb782a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:55.255344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768cdb5-b2e2-4228-ba0c-be268fb782a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:55.255845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768cdb5-b2e2-4228-ba0c-be268fb782a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:55.256285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768cdb5-b2e2-4228-ba0c-be268fb782a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:55.257772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768cdb5-b2e2-4228-ba0c-be268fb782a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:55.258968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 888/1773] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:04:54 2022] POST /identity/v3/projects => generated 381 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:55.260095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:55.262966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:55.263517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768cdb5-b2e2-4228-ba0c-be268fb782a2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:55.439343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04505cd2-be8b-43b7-aa1b-c1127aabbcfd admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:55.458873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 886/1774] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:54 2022] POST /identity/v3/users => generated 303 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:55.463284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768cdb5-b2e2-4228-ba0c-be268fb782a2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:55.469242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:55.470729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:55.471014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:55.471322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] PATH_INFO: `/v3/users/9abd4bb809fa40689e8b2ad77b936a0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:55.480267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abd4bb809fa40689e8b2ad77b936a0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:55.482583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:55.530219 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768cdb5-b2e2-4228-ba0c-be268fb782a2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:55.553973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:55.554689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:55.555103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:55.555439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:55.564158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:55.566155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:55.855011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 887/1775] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:04:55 2022] POST /identity/v3/users => generated 297 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:04:55.856133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:55.862675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:55.878344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abd4bb809fa40689e8b2ad77b936a0f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:55.880566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abd4bb809fa40689e8b2ad77b936a0f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:55.882140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abd4bb809fa40689e8b2ad77b936a0f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:55.911583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abd4bb809fa40689e8b2ad77b936a0f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:55.935816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abd4bb809fa40689e8b2ad77b936a0f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:56.034339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716c7c6-4b07-458c-8bb0-cbeb7bdccd2e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:56.046993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a07517-4e71-4915-8eee-3d1ef6ba99df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:56.048541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a07517-4e71-4915-8eee-3d1ef6ba99df service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:56.049242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a07517-4e71-4915-8eee-3d1ef6ba99df service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:56.050083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a07517-4e71-4915-8eee-3d1ef6ba99df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:56.053777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b6a07517-4e71-4915-8eee-3d1ef6ba99df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:56.068496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 889/1776] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:04:55 2022] DELETE /identity/v3/users/9abd4bb809fa40689e8b2ad77b936a0f => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:56.072745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:56.077228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:56.233928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:56.247651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a07517-4e71-4915-8eee-3d1ef6ba99df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:56.251128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a07517-4e71-4915-8eee-3d1ef6ba99df service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:56.282435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 888/1777] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:55 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:56.287269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01964d13-89bd-414a-86c9-bc449100c6c3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:56.288019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01964d13-89bd-414a-86c9-bc449100c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:56.288510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01964d13-89bd-414a-86c9-bc449100c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:56.317507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6ae0b-499a-4c15-9265-c1e7f2672fad admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:56.338315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:56.339562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:56.340063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:56.341634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:56.343728 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:56.349552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:56.380491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-01964d13-89bd-414a-86c9-bc449100c6c3 None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:56.380726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 890/1778] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:56.382230 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:56.385361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-01964d13-89bd-414a-86c9-bc449100c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:56.390496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:56.428354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:56.432368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcfa90-c181-49d8-94c0-9c6f1556dd60 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:56.471557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438176ec-cdc6-4d21-9bc9-8e3c5de2967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:56.475591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438176ec-cdc6-4d21-9bc9-8e3c5de2967f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:56.488714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438176ec-cdc6-4d21-9bc9-8e3c5de2967f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:56.489062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438176ec-cdc6-4d21-9bc9-8e3c5de2967f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:56.490286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438176ec-cdc6-4d21-9bc9-8e3c5de2967f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:56.492512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438176ec-cdc6-4d21-9bc9-8e3c5de2967f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:56.511450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 891/1779] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:56.513057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdfba2-9e62-4aa7-b586-c4a6cabb9014 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:56.513361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdfba2-9e62-4aa7-b586-c4a6cabb9014 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:56.514846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdfba2-9e62-4aa7-b586-c4a6cabb9014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:56.550876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-53cdfba2-9e62-4aa7-b586-c4a6cabb9014 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:56.553113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-53cdfba2-9e62-4aa7-b586-c4a6cabb9014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:56.566057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 892/1780] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:56.574976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdfba2-9e62-4aa7-b586-c4a6cabb9014 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:56.588216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdfba2-9e62-4aa7-b586-c4a6cabb9014 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:56.606221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4ab7c-c8ff-4701-97d2-3b6b1ae48a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:56.607256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ab7c-c8ff-4701-97d2-3b6b1ae48a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:56.607676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ab7c-c8ff-4701-97d2-3b6b1ae48a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:56.608008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ab7c-c8ff-4701-97d2-3b6b1ae48a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:56.608883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4ab7c-c8ff-4701-97d2-3b6b1ae48a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:56.614963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4ab7c-c8ff-4701-97d2-3b6b1ae48a99 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:56.656119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 893/1781] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:56.657605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4ab7c-c8ff-4701-97d2-3b6b1ae48a99 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:56.689979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4ab7c-c8ff-4701-97d2-3b6b1ae48a99 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:56.699538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 889/1782] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:04:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:56.708832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec2ca4-e823-4d2d-aeb2-e7ff342b7314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:56.710572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2ca4-e823-4d2d-aeb2-e7ff342b7314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:56.711157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2ca4-e823-4d2d-aeb2-e7ff342b7314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:56.711783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2ca4-e823-4d2d-aeb2-e7ff342b7314 admin admin] PATH_INFO: `/v3/projects/e3399131144f49ae97bb992f03c313a8/users/9aa3c867804b431db75b072e9c9ebca4/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:56.714966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01964d13-89bd-414a-86c9-bc449100c6c3 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:56.718047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-01964d13-89bd-414a-86c9-bc449100c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:56.772007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec2ca4-e823-4d2d-aeb2-e7ff342b7314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3399131144f49ae97bb992f03c313a8, user_id=9aa3c867804b431db75b072e9c9ebca4, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:56.779009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec2ca4-e823-4d2d-aeb2-e7ff342b7314 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:56.967265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01964d13-89bd-414a-86c9-bc449100c6c3 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:56.983300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 894/1783] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:56 2022] PUT /identity/v3/projects/e3399131144f49ae97bb992f03c313a8/users/9aa3c867804b431db75b072e9c9ebca4/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:56.988646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec2ca4-e823-4d2d-aeb2-e7ff342b7314 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:57.076516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:57.077467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:57.077865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:57.078183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:57.195922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:57.198936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:57.202322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec2ca4-e823-4d2d-aeb2-e7ff342b7314 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:57.209783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8714e6-87c1-4893-a108-6398f6ab6a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:57.211097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8714e6-87c1-4893-a108-6398f6ab6a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:57.212105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8714e6-87c1-4893-a108-6398f6ab6a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:57.212105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8714e6-87c1-4893-a108-6398f6ab6a91 admin admin] PATH_INFO: `/v3/projects/27bab7586bb548ff935b2ec2ed9acc06/users/745098eb1d8f48bdb025de871a3f7202/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:57.224744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 890/1784] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:57.225782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:57.229375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:57.235255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8714e6-87c1-4893-a108-6398f6ab6a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bab7586bb548ff935b2ec2ed9acc06, user_id=745098eb1d8f48bdb025de871a3f7202, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:57.240061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8714e6-87c1-4893-a108-6398f6ab6a91 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:57.256508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0200742-b16e-4cf9-af36-3759fae6e58d service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:57.266899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:57.268565 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:57.269238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:57.269894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] PATH_INFO: `/v3/projects/e0e98b34543f4717bbd76c6000db1b60` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:57.275498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e98b34543f4717bbd76c6000db1b60)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:57.282216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:57.389237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 895/1785] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:56 2022] PUT /identity/v3/projects/27bab7586bb548ff935b2ec2ed9acc06/users/745098eb1d8f48bdb025de871a3f7202/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:57.390666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8714e6-87c1-4893-a108-6398f6ab6a91 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:57.517477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:57.541388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e98b34543f4717bbd76c6000db1b60', 'request_id': 'req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:57.549883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e98b34543f4717bbd76c6000db1b60', 'request_id': 'req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:57.552764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8714e6-87c1-4893-a108-6398f6ab6a91 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:57.557464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e98b34543f4717bbd76c6000db1b60', 'request_id': 'req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:57.654823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:57.682634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:57.683666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:57.683958 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:57.684241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:57.689334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:57.712249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] The token cache is being invalidate because project e0e98b34543f4717bbd76c6000db1b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:04:57.712945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:57.713591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:57.715451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e47aa7b-f7ba-47ae-861b-4987f3afd2c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:57.720655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 891/1786] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:57 2022] DELETE /identity/v3/projects/e0e98b34543f4717bbd76c6000db1b60 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:57.722179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc789fec-d854-4261-a25a-a6082ca417ad None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:57.722475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc789fec-d854-4261-a25a-a6082ca417ad None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:57.722766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc789fec-d854-4261-a25a-a6082ca417ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:57.755017 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 896/1787] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:04:57.758863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:57.761333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:57.774630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dc789fec-d854-4261-a25a-a6082ca417ad None None] MFA Rules not processed for user `0beff646206e446386d91537df9eaa67`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:57.777424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dc789fec-d854-4261-a25a-a6082ca417ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:58.019545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d643df-19e7-414f-9dbe-c1c82c82caef admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:58.028097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 892/1788] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Jul 1 04:04:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:58.029666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefb8c-b0b7-4e35-acbd-354da8a96ff3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.029985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefb8c-b0b7-4e35-acbd-354da8a96ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.030304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefb8c-b0b7-4e35-acbd-354da8a96ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.050488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:58.054610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.055265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.056213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.070176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:58.082165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6faefb8c-b0b7-4e35-acbd-354da8a96ff3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:58.083700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6faefb8c-b0b7-4e35-acbd-354da8a96ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:58.097260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 893/1789] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:58.099455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb40dc2-3b0e-4a13-ba57-8110f1980c74 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.099944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb40dc2-3b0e-4a13-ba57-8110f1980c74 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.100357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb40dc2-3b0e-4a13-ba57-8110f1980c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.124307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0eb40dc2-3b0e-4a13-ba57-8110f1980c74 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:58.126560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0eb40dc2-3b0e-4a13-ba57-8110f1980c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:58.137746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 894/1790] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:58.139086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb40dc2-3b0e-4a13-ba57-8110f1980c74 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:58.140852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0eb40dc2-3b0e-4a13-ba57-8110f1980c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:58.157760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb40dc2-3b0e-4a13-ba57-8110f1980c74 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:58.166856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1711310-94e8-4154-a562-097923025983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:58.168144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1711310-94e8-4154-a562-097923025983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.168669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1711310-94e8-4154-a562-097923025983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.169251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1711310-94e8-4154-a562-097923025983 admin admin] PATH_INFO: `/v3/projects/e3399131144f49ae97bb992f03c313a8/users/9aa3c867804b431db75b072e9c9ebca4/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.173954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1711310-94e8-4154-a562-097923025983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3399131144f49ae97bb992f03c313a8, user_id=9aa3c867804b431db75b072e9c9ebca4, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:58.179173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1711310-94e8-4154-a562-097923025983 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:58.230337 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 895/1791] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:04:58 2022] PUT /identity/v3/projects/e3399131144f49ae97bb992f03c313a8/users/9aa3c867804b431db75b072e9c9ebca4/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:04:58.240082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1711310-94e8-4154-a562-097923025983 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:58.252169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a1711310-94e8-4154-a562-097923025983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:58.263377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:58.266922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:58.291984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 897/1792] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:04:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:04:58.297932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:58.297932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:58.314930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9972658-1066-4093-a7d0-b9ec73ab0bbe service swift] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:58.440399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1711310-94e8-4154-a562-097923025983 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:58.453298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5464de-85e1-4f00-97b4-a5b565e0cced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:58.456143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5464de-85e1-4f00-97b4-a5b565e0cced service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.456502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5464de-85e1-4f00-97b4-a5b565e0cced service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.458131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5464de-85e1-4f00-97b4-a5b565e0cced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.477935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5464de-85e1-4f00-97b4-a5b565e0cced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:58.480244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5464de-85e1-4f00-97b4-a5b565e0cced service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:58.519053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 896/1793] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:58.520108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5464de-85e1-4f00-97b4-a5b565e0cced service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:58.566808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5464de-85e1-4f00-97b4-a5b565e0cced service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:58.578251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9db63-0a0b-4ae4-8993-11b3bcde57fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:58.580375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9db63-0a0b-4ae4-8993-11b3bcde57fc service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.580846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9db63-0a0b-4ae4-8993-11b3bcde57fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.581299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9db63-0a0b-4ae4-8993-11b3bcde57fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.581562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bbe20-39ac-42a3-963e-fa180c556ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:58.582393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bbe20-39ac-42a3-963e-fa180c556ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.605550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bbe20-39ac-42a3-963e-fa180c556ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.605550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bbe20-39ac-42a3-963e-fa180c556ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.625766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9db63-0a0b-4ae4-8993-11b3bcde57fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:58.629964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9db63-0a0b-4ae4-8993-11b3bcde57fc service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:58.634873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bbe20-39ac-42a3-963e-fa180c556ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:58.636719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bbe20-39ac-42a3-963e-fa180c556ff6 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:58.673108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:58.678024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 897/1794] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:58.683087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bbe20-39ac-42a3-963e-fa180c556ff6 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:58.715847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 898/1795] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:58.717628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b8bba-0474-465b-b955-8301da654db6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.717967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b8bba-0474-465b-b955-8301da654db6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.718284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b8bba-0474-465b-b955-8301da654db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.778346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bbe20-39ac-42a3-963e-fa180c556ff6 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:58.798295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb4431-9954-488f-ae4b-5825d3fc6117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:58.802346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb4431-9954-488f-ae4b-5825d3fc6117 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:58.802346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb4431-9954-488f-ae4b-5825d3fc6117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:58.802346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb4431-9954-488f-ae4b-5825d3fc6117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:58.837696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-de6b8bba-0474-465b-b955-8301da654db6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:58.838710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-de6b8bba-0474-465b-b955-8301da654db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:59.018991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb4431-9954-488f-ae4b-5825d3fc6117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:59.037865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb4431-9954-488f-ae4b-5825d3fc6117 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:59.071918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 898/1796] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:04:59.073812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a6f8d-8163-4c68-8ba4-8842608b9ba7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:59.074198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a6f8d-8163-4c68-8ba4-8842608b9ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:59.074503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a6f8d-8163-4c68-8ba4-8842608b9ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:59.169374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:04:59.198316 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 899/1797] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:59.207177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b8bba-0474-465b-b955-8301da654db6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:59.218583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-de6b8bba-0474-465b-b955-8301da654db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:59.252693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-311a6f8d-8163-4c68-8ba4-8842608b9ba7 None None] MFA Rules not processed for user `9aa3c867804b431db75b072e9c9ebca4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:04:59.256877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-311a6f8d-8163-4c68-8ba4-8842608b9ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:59.291683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b8bba-0474-465b-b955-8301da654db6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:59.324723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:59.330628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:59.331010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:59.331361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] PATH_INFO: `/v3/projects/c79c8683f6614a9a9d883be0553d6b7e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:59.350645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79c8683f6614a9a9d883be0553d6b7e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:59.353821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:59.540143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 899/1798] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:04:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:04:59.540143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a6f8d-8163-4c68-8ba4-8842608b9ba7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:59.584958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a6f8d-8163-4c68-8ba4-8842608b9ba7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:04:59.596430 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:04:59.598079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:04:59.599182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:04:59.600690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:04:59.604427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:04:59.605080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:04:59.622430 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:04:59.667392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 900/1799] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:04:59.667392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:04:59.667392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:04:59.856109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79c8683f6614a9a9d883be0553d6b7e', 'request_id': 'req-829705ea-c10f-4c10-851b-77f4a60e4150'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:59.866800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79c8683f6614a9a9d883be0553d6b7e', 'request_id': 'req-829705ea-c10f-4c10-851b-77f4a60e4150'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:04:59.877789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3ffa9-8942-436b-a061-eaa30a0e3b96 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:00.029557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce7d70-f1c0-4457-bfa6-221531e82c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:00.030720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7d70-f1c0-4457-bfa6-221531e82c1d service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:00.031306 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7d70-f1c0-4457-bfa6-221531e82c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:00.032211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7d70-f1c0-4457-bfa6-221531e82c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:00.174874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce7d70-f1c0-4457-bfa6-221531e82c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:00.176360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce7d70-f1c0-4457-bfa6-221531e82c1d service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:00.199248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 901/1800] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:04:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:00.200296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce7d70-f1c0-4457-bfa6-221531e82c1d service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:00.408779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce7d70-f1c0-4457-bfa6-221531e82c1d service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:00.414293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:00.415325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:00.415583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:00.415753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:00.416647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:00.418990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:01.739109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79c8683f6614a9a9d883be0553d6b7e', 'request_id': 'req-829705ea-c10f-4c10-851b-77f4a60e4150'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:01.762987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 902/1801] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:00 2022] POST /identity/v3/projects => generated 369 bytes in 1560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:01.767372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] The token cache is being invalidate because project c79c8683f6614a9a9d883be0553d6b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:01.767933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:01.768974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:01.771137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 900/1802] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:04:59 2022] DELETE /identity/v3/projects/c79c8683f6614a9a9d883be0553d6b7e => generated 0 bytes in 2577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:01.772500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:01.774212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:01.779266 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:01.792289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:01.799057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829705ea-c10f-4c10-851b-77f4a60e4150 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:01.856946 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:02.019886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bb805-8cac-499e-8d36-92ab10979bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:02.021053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb805-8cac-499e-8d36-92ab10979bea admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.021564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb805-8cac-499e-8d36-92ab10979bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.022077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb805-8cac-499e-8d36-92ab10979bea admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.023446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bb805-8cac-499e-8d36-92ab10979bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:02.028565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bb805-8cac-499e-8d36-92ab10979bea admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:02.045490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699e10b-4099-4f92-a919-f28d9cd95160 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:02.061798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:02.062704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.063571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.064813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.070596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:02.166311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 901/1803] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:01 2022] POST /identity/v3/users => generated 285 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:02.169258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e2f06-6f0a-475b-ac94-3e3bffd31419 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.185146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e2f06-6f0a-475b-ac94-3e3bffd31419 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.185999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e2f06-6f0a-475b-ac94-3e3bffd31419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.268101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d25e2f06-6f0a-475b-ac94-3e3bffd31419 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:02.270848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d25e2f06-6f0a-475b-ac94-3e3bffd31419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:02.287795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 902/1804] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:02.289388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e2f06-6f0a-475b-ac94-3e3bffd31419 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:02.308363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e2f06-6f0a-475b-ac94-3e3bffd31419 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:02.323319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c15cc-c025-404e-933d-852f6d87d227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:02.326117 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c15cc-c025-404e-933d-852f6d87d227 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.329465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c15cc-c025-404e-933d-852f6d87d227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.329465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c15cc-c025-404e-933d-852f6d87d227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.329465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c15cc-c025-404e-933d-852f6d87d227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:02.334136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c15cc-c025-404e-933d-852f6d87d227 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:02.367796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 903/1805] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:02.391806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec449cf0-ff6d-4aca-8db4-247200937a4e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.391806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec449cf0-ff6d-4aca-8db4-247200937a4e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.391806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec449cf0-ff6d-4aca-8db4-247200937a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.433306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ec449cf0-ff6d-4aca-8db4-247200937a4e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:02.434579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ec449cf0-ff6d-4aca-8db4-247200937a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:02.455649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 904/1806] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:02.456160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec449cf0-ff6d-4aca-8db4-247200937a4e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:02.488372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:02.489692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec449cf0-ff6d-4aca-8db4-247200937a4e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:02.490708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:02.504065 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ec2c3-68fb-467e-9e2e-79bb5aad2f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:02.504771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec2c3-68fb-467e-9e2e-79bb5aad2f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.505064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec2c3-68fb-467e-9e2e-79bb5aad2f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.505338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec2c3-68fb-467e-9e2e-79bb5aad2f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.506103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ec2c3-68fb-467e-9e2e-79bb5aad2f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:02.507832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ec2c3-68fb-467e-9e2e-79bb5aad2f72 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:02.569759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 905/1807] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:02.569759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ec2c3-68fb-467e-9e2e-79bb5aad2f72 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:02.570161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 903/1808] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 804 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:02.570161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:02.595315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ec2c3-68fb-467e-9e2e-79bb5aad2f72 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:02.609617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587c77-d29e-4d28-84d6-17d3410cf43e service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:02.614542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:02.615578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.615930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.616295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.617122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:02.617579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:02.629258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:02.629641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca8979-b0f7-4c7c-a2e3-a8c5c95e69c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:02.632276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca8979-b0f7-4c7c-a2e3-a8c5c95e69c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.632633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca8979-b0f7-4c7c-a2e3-a8c5c95e69c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.632930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca8979-b0f7-4c7c-a2e3-a8c5c95e69c1 admin admin] PATH_INFO: `/v3/projects/bfd70e00125c4f589c84ead04fb99cec/users/39896f5b78ab46b8a499ed4acbfacc96/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.674324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 906/1809] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:02.678637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:02.723611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca8979-b0f7-4c7c-a2e3-a8c5c95e69c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd70e00125c4f589c84ead04fb99cec, user_id=39896f5b78ab46b8a499ed4acbfacc96, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:02.736363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d42288-2220-40e7-bccd-9dfabab1a688 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:02.740139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca8979-b0f7-4c7c-a2e3-a8c5c95e69c1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:02.759016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ce2e9-100b-479d-abf4-b8b4c5102da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:02.760496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce2e9-100b-479d-abf4-b8b4c5102da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:02.760852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce2e9-100b-479d-abf4-b8b4c5102da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:02.761141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce2e9-100b-479d-abf4-b8b4c5102da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:02.763791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ce2e9-100b-479d-abf4-b8b4c5102da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:02.771110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ce2e9-100b-479d-abf4-b8b4c5102da2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:03.400802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 904/1810] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:02 2022] PUT /identity/v3/projects/bfd70e00125c4f589c84ead04fb99cec/users/39896f5b78ab46b8a499ed4acbfacc96/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:03.414442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134707b6-fcc1-43e3-9490-6b584500d28a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:03.418164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134707b6-fcc1-43e3-9490-6b584500d28a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:03.418164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134707b6-fcc1-43e3-9490-6b584500d28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:03.468757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-134707b6-fcc1-43e3-9490-6b584500d28a None None] MFA Rules not processed for user `39896f5b78ab46b8a499ed4acbfacc96`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:03.470838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-134707b6-fcc1-43e3-9490-6b584500d28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:03.508872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 907/1811] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:02 2022] POST /identity/v3/projects => generated 381 bytes in 831 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:03.513951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ce2e9-100b-479d-abf4-b8b4c5102da2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:03.758817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ce2e9-100b-479d-abf4-b8b4c5102da2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:03.780300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbb51b-5169-48e4-a29e-bc16ac607fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:03.783196 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbb51b-5169-48e4-a29e-bc16ac607fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:03.783665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbb51b-5169-48e4-a29e-bc16ac607fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:03.785082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbb51b-5169-48e4-a29e-bc16ac607fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:03.786137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbb51b-5169-48e4-a29e-bc16ac607fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:03.788583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbb51b-5169-48e4-a29e-bc16ac607fa3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:03.889232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 905/1812] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:03.896398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134707b6-fcc1-43e3-9490-6b584500d28a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:03.937545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 908/1813] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:03 2022] POST /identity/v3/users => generated 291 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:03.945406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbb51b-5169-48e4-a29e-bc16ac607fa3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:03.988424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbb51b-5169-48e4-a29e-bc16ac607fa3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:04.012132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201db0-c8bc-4d5e-a719-817e002cee13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:04.012851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201db0-c8bc-4d5e-a719-817e002cee13 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:04.013138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201db0-c8bc-4d5e-a719-817e002cee13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:04.013417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201db0-c8bc-4d5e-a719-817e002cee13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:04.014186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a201db0-c8bc-4d5e-a719-817e002cee13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:04.026693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a201db0-c8bc-4d5e-a719-817e002cee13 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:04.061831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 909/1814] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:04.064169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201db0-c8bc-4d5e-a719-817e002cee13 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:04.097029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201db0-c8bc-4d5e-a719-817e002cee13 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:04.142093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134707b6-fcc1-43e3-9490-6b584500d28a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:04.150049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eb7ce-8af0-4e4a-ad0c-cdd10ce6d7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:04.150812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eb7ce-8af0-4e4a-ad0c-cdd10ce6d7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:04.151125 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eb7ce-8af0-4e4a-ad0c-cdd10ce6d7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:04.151621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eb7ce-8af0-4e4a-ad0c-cdd10ce6d7ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:04.152516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43eb7ce-8af0-4e4a-ad0c-cdd10ce6d7ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:04.154803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43eb7ce-8af0-4e4a-ad0c-cdd10ce6d7ab admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:04.169402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91170e01-b620-4574-b224-7ba367c3c97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:04.174352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91170e01-b620-4574-b224-7ba367c3c97b admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:04.174925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91170e01-b620-4574-b224-7ba367c3c97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:04.179632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91170e01-b620-4574-b224-7ba367c3c97b admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:04.181357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91170e01-b620-4574-b224-7ba367c3c97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:04.189436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91170e01-b620-4574-b224-7ba367c3c97b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:04.317872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 910/1815] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:04.320092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91170e01-b620-4574-b224-7ba367c3c97b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:04.343410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91170e01-b620-4574-b224-7ba367c3c97b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:04.356434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cca4b-7105-4e58-8814-7b68abc33a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:04.358914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cca4b-7105-4e58-8814-7b68abc33a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:04.361145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cca4b-7105-4e58-8814-7b68abc33a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:04.361713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cca4b-7105-4e58-8814-7b68abc33a6d admin admin] PATH_INFO: `/v3/projects/8699f97b8d9843efbcf5521bfeae30b0/users/a07facc432e744ddbaafba4134ec73df/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:04.388295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 906/1816] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:03 2022] POST /identity/v3/projects => generated 367 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:04.389749 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eb7ce-8af0-4e4a-ad0c-cdd10ce6d7ab admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:04.454118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cca4b-7105-4e58-8814-7b68abc33a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8699f97b8d9843efbcf5521bfeae30b0, user_id=a07facc432e744ddbaafba4134ec73df, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:04.460441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cca4b-7105-4e58-8814-7b68abc33a6d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:04.576205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eb7ce-8af0-4e4a-ad0c-cdd10ce6d7ab admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:04.618473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fc86c-79bc-4b16-964c-1082c46b57f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:04.619545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc86c-79bc-4b16-964c-1082c46b57f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:04.620062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc86c-79bc-4b16-964c-1082c46b57f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:04.620577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc86c-79bc-4b16-964c-1082c46b57f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:04.621939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fc86c-79bc-4b16-964c-1082c46b57f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:04.626354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fc86c-79bc-4b16-964c-1082c46b57f4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:04.708728 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 911/1817] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:04 2022] PUT /identity/v3/projects/8699f97b8d9843efbcf5521bfeae30b0/users/a07facc432e744ddbaafba4134ec73df/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:04.751004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f975b-8244-473f-a96a-975d7688b5e5 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:04.751811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f975b-8244-473f-a96a-975d7688b5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:04.752308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f975b-8244-473f-a96a-975d7688b5e5 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:04.761309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 912/1818] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:05:04 2022] GET /identity => generated 270 bytes in 52 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:05:04.765835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f0af-8c62-4b96-a31f-73b49fa16963 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:04.770773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f0af-8c62-4b96-a31f-73b49fa16963 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:04.777803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f0af-8c62-4b96-a31f-73b49fa16963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:04.986718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6121f0af-8c62-4b96-a31f-73b49fa16963 None None] MFA Rules not processed for user `a07facc432e744ddbaafba4134ec73df`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:04.989367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6121f0af-8c62-4b96-a31f-73b49fa16963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:05.035034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 907/1819] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:04 2022] POST /identity/v3/users => generated 290 bytes in 646 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:05.039814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875214d-ef51-4e2f-a1e6-373f9897187b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:05.041567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875214d-ef51-4e2f-a1e6-373f9897187b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:05.042130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875214d-ef51-4e2f-a1e6-373f9897187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:05.086803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0875214d-ef51-4e2f-a1e6-373f9897187b None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:05.089345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0875214d-ef51-4e2f-a1e6-373f9897187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:05.351597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 913/1820] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 589 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:05.353097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121f0af-8c62-4b96-a31f-73b49fa16963 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:05.354500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 908/1821] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:05:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:05.362590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8288f-fa84-486d-9b7d-b6b376219fc6 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:05.364316 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8288f-fa84-486d-9b7d-b6b376219fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:05.364849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8288f-fa84-486d-9b7d-b6b376219fc6 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:05.370015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 909/1822] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:05:05 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:05:05.377988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8288f-fa84-486d-9b7d-b6b376219fc6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:05.667151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8288f-fa84-486d-9b7d-b6b376219fc6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:05.669162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121f0af-8c62-4b96-a31f-73b49fa16963 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:05.684069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60724672-efbf-4c31-a996-6f82d7fa74bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:05.685036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596a4eb-8983-48ed-9377-189e5cae5f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:05.696097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724672-efbf-4c31-a996-6f82d7fa74bd admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:05.696391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724672-efbf-4c31-a996-6f82d7fa74bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:05.696671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724672-efbf-4c31-a996-6f82d7fa74bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:05.697452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60724672-efbf-4c31-a996-6f82d7fa74bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:05.704374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596a4eb-8983-48ed-9377-189e5cae5f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:05.704877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596a4eb-8983-48ed-9377-189e5cae5f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:05.705056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596a4eb-8983-48ed-9377-189e5cae5f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:05.705817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596a4eb-8983-48ed-9377-189e5cae5f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:05.719398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60724672-efbf-4c31-a996-6f82d7fa74bd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:05.721635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596a4eb-8983-48ed-9377-189e5cae5f07 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:05.768570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 914/1823] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:05.779250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8d8bb-0ca2-4f31-accc-c8248e2a1ece None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:05.779896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8d8bb-0ca2-4f31-accc-c8248e2a1ece None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:05.780387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8d8bb-0ca2-4f31-accc-c8248e2a1ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:05.903856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 910/1824] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:05 2022] POST /identity/v3/projects => generated 381 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:05.904797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596a4eb-8983-48ed-9377-189e5cae5f07 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:05.908844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-67b8d8bb-0ca2-4f31-accc-c8248e2a1ece None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:05.913223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-67b8d8bb-0ca2-4f31-accc-c8248e2a1ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:06.075191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:06.226083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596a4eb-8983-48ed-9377-189e5cae5f07 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:06.236171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbed47-865b-4f88-abda-973425a9c56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:06.237506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbed47-865b-4f88-abda-973425a9c56a admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:06.237952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbed47-865b-4f88-abda-973425a9c56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:06.238395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbed47-865b-4f88-abda-973425a9c56a admin admin] PATH_INFO: `/v3/projects/1c2334fa740b46e0b9e20d5b9192501b/users/db1b22b6bcd2452a8effb2dc47369451/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:06.292104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbed47-865b-4f88-abda-973425a9c56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2334fa740b46e0b9e20d5b9192501b, user_id=db1b22b6bcd2452a8effb2dc47369451, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:06.295126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbed47-865b-4f88-abda-973425a9c56a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:06.412975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 915/1825] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:05:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 648 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:06.415792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8d8bb-0ca2-4f31-accc-c8248e2a1ece None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:06.417855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 911/1826] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:05 2022] PUT /identity/v3/projects/1c2334fa740b46e0b9e20d5b9192501b/users/db1b22b6bcd2452a8effb2dc47369451/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:06.418916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbed47-865b-4f88-abda-973425a9c56a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:06.513866 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8d8bb-0ca2-4f31-accc-c8248e2a1ece None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:06.652759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbed47-865b-4f88-abda-973425a9c56a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:06.658488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:06.659166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:06.659477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:06.659760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:06.660529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:06.662213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:06.731529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 912/1827] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:06.738847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:06.741482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:06.854970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563935d-e246-460d-8310-9904bf79a7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:06.858763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563935d-e246-460d-8310-9904bf79a7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:06.859595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563935d-e246-460d-8310-9904bf79a7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:06.865226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563935d-e246-460d-8310-9904bf79a7f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:06.865226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563935d-e246-460d-8310-9904bf79a7f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:06.868991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563935d-e246-460d-8310-9904bf79a7f1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:06.998750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a63b54-ee94-40b2-9144-aebe841e9247 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:07.024497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d2a98-b181-4f1f-90db-7e5929f264e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:07.025219 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d2a98-b181-4f1f-90db-7e5929f264e7 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:07.025571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d2a98-b181-4f1f-90db-7e5929f264e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:07.025860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d2a98-b181-4f1f-90db-7e5929f264e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:07.052266 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d2a98-b181-4f1f-90db-7e5929f264e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:07.053959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d2a98-b181-4f1f-90db-7e5929f264e7 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:07.062951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 916/1828] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:06 2022] POST /identity/v3/users => generated 297 bytes in 649 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:07.064635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563935d-e246-460d-8310-9904bf79a7f1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:07.084688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563935d-e246-460d-8310-9904bf79a7f1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:07.103759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d1ed9-4424-4902-8a33-78e1f249fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:07.106465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d1ed9-4424-4902-8a33-78e1f249fb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:07.106909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d1ed9-4424-4902-8a33-78e1f249fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:07.108021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d1ed9-4424-4902-8a33-78e1f249fb8a admin admin] PATH_INFO: `/v3/projects/1c2334fa740b46e0b9e20d5b9192501b/users/db1b22b6bcd2452a8effb2dc47369451/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:07.114116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d1ed9-4424-4902-8a33-78e1f249fb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2334fa740b46e0b9e20d5b9192501b, user_id=db1b22b6bcd2452a8effb2dc47369451, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:07.118954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d1ed9-4424-4902-8a33-78e1f249fb8a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:07.126124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 913/1829] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:07.128000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d2a98-b181-4f1f-90db-7e5929f264e7 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:07.166178 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 917/1830] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:07 2022] PUT /identity/v3/projects/1c2334fa740b46e0b9e20d5b9192501b/users/db1b22b6bcd2452a8effb2dc47369451/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:07.170481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7a12-1c99-4bc4-8af3-3e440bf5a3b0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:07.170915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7a12-1c99-4bc4-8af3-3e440bf5a3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:07.172741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d2a98-b181-4f1f-90db-7e5929f264e7 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:07.173116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7a12-1c99-4bc4-8af3-3e440bf5a3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:07.211680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:07.263158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fa4c7a12-1c99-4bc4-8af3-3e440bf5a3b0 None None] MFA Rules not processed for user `db1b22b6bcd2452a8effb2dc47369451`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:07.272940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fa4c7a12-1c99-4bc4-8af3-3e440bf5a3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:07.517324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 918/1831] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:07.517324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7a12-1c99-4bc4-8af3-3e440bf5a3b0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:07.525880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0042ea1-1f07-4b7f-967b-94edafd5a526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:07.527143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0042ea1-1f07-4b7f-967b-94edafd5a526 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:07.532955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0042ea1-1f07-4b7f-967b-94edafd5a526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:07.534912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0042ea1-1f07-4b7f-967b-94edafd5a526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:07.539215 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0042ea1-1f07-4b7f-967b-94edafd5a526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:07.546118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0042ea1-1f07-4b7f-967b-94edafd5a526 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:07.570887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 914/1832] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:07.570887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0042ea1-1f07-4b7f-967b-94edafd5a526 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:07.772647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0042ea1-1f07-4b7f-967b-94edafd5a526 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:07.785305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661dab9-9c82-430e-bba1-34eb980a18c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:07.787343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661dab9-9c82-430e-bba1-34eb980a18c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:07.787667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661dab9-9c82-430e-bba1-34eb980a18c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:07.787930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661dab9-9c82-430e-bba1-34eb980a18c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:07.793993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7a12-1c99-4bc4-8af3-3e440bf5a3b0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:07.857076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661dab9-9c82-430e-bba1-34eb980a18c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:07.857076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661dab9-9c82-430e-bba1-34eb980a18c4 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:07.861148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dead3-809b-4c40-9bb7-67ef40ed0930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:07.861148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dead3-809b-4c40-9bb7-67ef40ed0930 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:07.861148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dead3-809b-4c40-9bb7-67ef40ed0930 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:07.861148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dead3-809b-4c40-9bb7-67ef40ed0930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:07.864577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dead3-809b-4c40-9bb7-67ef40ed0930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:07.869965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dead3-809b-4c40-9bb7-67ef40ed0930 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:07.877547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 915/1833] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:07 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:07.879398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661dab9-9c82-430e-bba1-34eb980a18c4 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:07.926704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 919/1834] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:07.954565 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661dab9-9c82-430e-bba1-34eb980a18c4 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:08.015016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73715e7-55ed-4a6b-a469-676e0141e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:08.021238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73715e7-55ed-4a6b-a469-676e0141e9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:08.022606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73715e7-55ed-4a6b-a469-676e0141e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:08.027361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73715e7-55ed-4a6b-a469-676e0141e9f8 admin admin] PATH_INFO: `/v3/projects/3c397a12ca9c47a0ac4d78e016c93bbf/users/32a70cdb408c4950a55696edf2921cfc/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:08.116107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73715e7-55ed-4a6b-a469-676e0141e9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c397a12ca9c47a0ac4d78e016c93bbf, user_id=32a70cdb408c4950a55696edf2921cfc, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:08.116107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73715e7-55ed-4a6b-a469-676e0141e9f8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:08.284129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 916/1835] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:07 2022] PUT /identity/v3/projects/3c397a12ca9c47a0ac4d78e016c93bbf/users/32a70cdb408c4950a55696edf2921cfc/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:08.286004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dead3-809b-4c40-9bb7-67ef40ed0930 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:08.355686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dead3-809b-4c40-9bb7-67ef40ed0930 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:08.572243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a90381-1fcf-456a-93bb-1026207d31a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:08.573253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a90381-1fcf-456a-93bb-1026207d31a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:08.573253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a90381-1fcf-456a-93bb-1026207d31a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:08.573566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a90381-1fcf-456a-93bb-1026207d31a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:08.574561 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a90381-1fcf-456a-93bb-1026207d31a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:08.576448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a90381-1fcf-456a-93bb-1026207d31a1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:08.601128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 920/1836] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:08.608862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73715e7-55ed-4a6b-a469-676e0141e9f8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:08.639499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73715e7-55ed-4a6b-a469-676e0141e9f8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:08.664959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:08.667057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:08.667695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:08.668227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] PATH_INFO: `/v3/projects/3c397a12ca9c47a0ac4d78e016c93bbf/users/32a70cdb408c4950a55696edf2921cfc/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:08.681470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c397a12ca9c47a0ac4d78e016c93bbf, user_id=32a70cdb408c4950a55696edf2921cfc, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:08.688849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:08.741559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 917/1837] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:08 2022] PUT /identity/v3/projects/3c397a12ca9c47a0ac4d78e016c93bbf/users/32a70cdb408c4950a55696edf2921cfc/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:08.749295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec555b7-f3d9-4bcb-bedd-3a7c491689b4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:08.749583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec555b7-f3d9-4bcb-bedd-3a7c491689b4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:08.749891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec555b7-f3d9-4bcb-bedd-3a7c491689b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:08.813919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-dec555b7-f3d9-4bcb-bedd-3a7c491689b4 None None] MFA Rules not processed for user `32a70cdb408c4950a55696edf2921cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:08.823662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-dec555b7-f3d9-4bcb-bedd-3a7c491689b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:09.011693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 921/1838] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:09.019691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:09.024108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:09.170103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf5237-aaa4-4f15-8767-37cc72d301e1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:09.187571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:09.189144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:09.189627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:09.191204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:09.219983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:09.224304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:09.240302 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 918/1839] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:09.761029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec555b7-f3d9-4bcb-bedd-3a7c491689b4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:09.784684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec555b7-f3d9-4bcb-bedd-3a7c491689b4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:09.804214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67e38e-2dd6-4092-876e-4947d85894f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:09.804976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e38e-2dd6-4092-876e-4947d85894f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:09.805266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e38e-2dd6-4092-876e-4947d85894f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:09.805548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e38e-2dd6-4092-876e-4947d85894f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:09.928052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67e38e-2dd6-4092-876e-4947d85894f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:09.929811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67e38e-2dd6-4092-876e-4947d85894f5 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:09.947621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 922/1840] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:10.067750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:10.069526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:10.214852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fae77-a3f8-484f-95d1-16c63e219146 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:10.215453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fae77-a3f8-484f-95d1-16c63e219146 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:10.215874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fae77-a3f8-484f-95d1-16c63e219146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:10.362996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-074fae77-a3f8-484f-95d1-16c63e219146 None None] MFA Rules not processed for user `745098eb1d8f48bdb025de871a3f7202`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:10.379862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb177ec9-cbc3-4755-84d2-b1e21c7c3048 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:10.391570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-074fae77-a3f8-484f-95d1-16c63e219146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:10.414138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:10.415138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:10.415678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:10.416186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] PATH_INFO: `/v3/users/2bed8e3a24a6463d98fd793e72b19939` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:10.418317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bed8e3a24a6463d98fd793e72b19939)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:10.420380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:10.594938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 923/1841] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:10.611273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fae77-a3f8-484f-95d1-16c63e219146 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:10.612747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-074fae77-a3f8-484f-95d1-16c63e219146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:10.667385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fae77-a3f8-484f-95d1-16c63e219146 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:10.729695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:10.730403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:10.730684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:10.730956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:10.731724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:10.733749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:10.751389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bed8e3a24a6463d98fd793e72b19939'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:10.758016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bed8e3a24a6463d98fd793e72b19939'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:10.849889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bed8e3a24a6463d98fd793e72b19939'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:10.870303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bed8e3a24a6463d98fd793e72b19939'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:10.882465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ac2859b-0f2a-4720-9545-6745b032baec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bed8e3a24a6463d98fd793e72b19939'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:10.919019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 924/1842] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:10 2022] POST /identity/v3/projects => generated 403 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:10.932734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:10.935659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:10.980437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:11.006558 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 919/1843] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:10 2022] DELETE /identity/v3/users/2bed8e3a24a6463d98fd793e72b19939 => generated 0 bytes in 939 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:11.013798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d324d16-13f2-4214-ba48-8e776685b5d7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:11.014561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d324d16-13f2-4214-ba48-8e776685b5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:11.016874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d324d16-13f2-4214-ba48-8e776685b5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:11.074827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3d324d16-13f2-4214-ba48-8e776685b5d7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:11.077972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3d324d16-13f2-4214-ba48-8e776685b5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:11.125131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6aae2-6f3c-4cc1-8363-5ce7c2e8f02d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:11.310103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6dba8-c414-4492-a1b4-afc502beccb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:11.320455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6dba8-c414-4492-a1b4-afc502beccb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:11.330279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6dba8-c414-4492-a1b4-afc502beccb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:11.333572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6dba8-c414-4492-a1b4-afc502beccb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:11.359650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6dba8-c414-4492-a1b4-afc502beccb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:11.379569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6dba8-c414-4492-a1b4-afc502beccb5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:11.427985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 920/1844] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:11.581722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d324d16-13f2-4214-ba48-8e776685b5d7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:11.584637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3d324d16-13f2-4214-ba48-8e776685b5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:11.761062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d324d16-13f2-4214-ba48-8e776685b5d7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:11.766607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4778f83-2f00-40bb-8718-ded9c04e0cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:11.767547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4778f83-2f00-40bb-8718-ded9c04e0cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:11.767954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4778f83-2f00-40bb-8718-ded9c04e0cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:11.768395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4778f83-2f00-40bb-8718-ded9c04e0cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:11.807047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4778f83-2f00-40bb-8718-ded9c04e0cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:11.814047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4778f83-2f00-40bb-8718-ded9c04e0cf9 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:11.851923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 921/1845] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:11.993412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 925/1846] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:10 2022] POST /identity/v3/users => generated 308 bytes in 1062 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:11.994246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4778f83-2f00-40bb-8718-ded9c04e0cf9 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:12.018501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4778f83-2f00-40bb-8718-ded9c04e0cf9 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:12.045020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ece9ec-8927-43db-adae-73e16938ce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:12.045815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ece9ec-8927-43db-adae-73e16938ce36 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:12.046096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ece9ec-8927-43db-adae-73e16938ce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:12.052660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ece9ec-8927-43db-adae-73e16938ce36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:12.053613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ece9ec-8927-43db-adae-73e16938ce36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:12.063152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ece9ec-8927-43db-adae-73e16938ce36 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:12.129489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 922/1847] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:12.135388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6dba8-c414-4492-a1b4-afc502beccb5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:12.159705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6dba8-c414-4492-a1b4-afc502beccb5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:12.186236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dfd41-6efb-4aee-8849-894dcee4831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:12.188917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dfd41-6efb-4aee-8849-894dcee4831a admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:12.189436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dfd41-6efb-4aee-8849-894dcee4831a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:12.189952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dfd41-6efb-4aee-8849-894dcee4831a admin admin] PATH_INFO: `/v3/projects/cbdb6ffadf8144f5bf9319df3219d0bd/users/fb0cf7d41be641a493d8b56e7d2eaf40/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:12.287188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26dfd41-6efb-4aee-8849-894dcee4831a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdb6ffadf8144f5bf9319df3219d0bd, user_id=fb0cf7d41be641a493d8b56e7d2eaf40, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:12.290927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26dfd41-6efb-4aee-8849-894dcee4831a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:12.411755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 926/1848] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:12 2022] PUT /identity/v3/projects/cbdb6ffadf8144f5bf9319df3219d0bd/users/fb0cf7d41be641a493d8b56e7d2eaf40/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:12.413702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ece9ec-8927-43db-adae-73e16938ce36 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:12.447454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ece9ec-8927-43db-adae-73e16938ce36 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:12.637659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:12.674739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:12.676256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:12.676797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:12.677251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:12.679545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:12.686101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:12.730467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 923/1849] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:12.731785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dfd41-6efb-4aee-8849-894dcee4831a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:12.752741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dfd41-6efb-4aee-8849-894dcee4831a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:12.760690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:12.761723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:12.762009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:12.762339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] PATH_INFO: `/v3/projects/cbdb6ffadf8144f5bf9319df3219d0bd/users/fb0cf7d41be641a493d8b56e7d2eaf40/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:12.764672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdb6ffadf8144f5bf9319df3219d0bd, user_id=fb0cf7d41be641a493d8b56e7d2eaf40, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:12.768060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:12.782404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 927/1850] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:12 2022] PUT /identity/v3/projects/cbdb6ffadf8144f5bf9319df3219d0bd/users/fb0cf7d41be641a493d8b56e7d2eaf40/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:13.214806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:13.217687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:13.245416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b40a-738b-4609-b26b-ffb6235fac8b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:13.357287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:13.358625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:13.359159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:13.360199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] PATH_INFO: `/v3/projects/666e4d296dfd4500afdf68e39926b8d6` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:13.361822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] RBAC: Authorizing `identity:delete_project(project_id=666e4d296dfd4500afdf68e39926b8d6)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:13.364072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:13.549151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666e4d296dfd4500afdf68e39926b8d6', 'request_id': 'req-da290601-1a46-4e0e-bc4c-70867065fcab'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:13.564212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666e4d296dfd4500afdf68e39926b8d6', 'request_id': 'req-da290601-1a46-4e0e-bc4c-70867065fcab'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:13.581904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666e4d296dfd4500afdf68e39926b8d6', 'request_id': 'req-da290601-1a46-4e0e-bc4c-70867065fcab'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:13.721058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] The token cache is being invalidate because project 666e4d296dfd4500afdf68e39926b8d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:13.721058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:13.721577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:13.725085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 924/1851] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:13 2022] DELETE /identity/v3/projects/666e4d296dfd4500afdf68e39926b8d6 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:13.730745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:13.735661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:13.770742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527de14-5b5c-40af-b45e-80369a8d2575 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:13.912505 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:13.916295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:13.916964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:13.917578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] PATH_INFO: `/v3/users/9aa3c867804b431db75b072e9c9ebca4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:13.921620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa3c867804b431db75b072e9c9ebca4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:13.925766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:14.124903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:14.318825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa3c867804b431db75b072e9c9ebca4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:14.327350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa3c867804b431db75b072e9c9ebca4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:14.327350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa3c867804b431db75b072e9c9ebca4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:14.351821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa3c867804b431db75b072e9c9ebca4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:14.370862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-db366e1f-9ddb-45ca-a538-32442c6b2acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa3c867804b431db75b072e9c9ebca4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:14.474385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 928/1852] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:13 2022] DELETE /identity/v3/users/9aa3c867804b431db75b072e9c9ebca4 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:14.995036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:15.786179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:16.138383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:16.219654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:16.239766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da290601-1a46-4e0e-bc4c-70867065fcab admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:16.358108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:16.360229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:16.360761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:16.361208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] PATH_INFO: `/v3/projects/e3399131144f49ae97bb992f03c313a8` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:16.364283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3399131144f49ae97bb992f03c313a8)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:16.368463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:16.503042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6c18-3f1e-40f3-9aab-db487d3bdd77 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:16.503975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6c18-3f1e-40f3-9aab-db487d3bdd77 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:16.504157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6c18-3f1e-40f3-9aab-db487d3bdd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:16.543447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-916c6c18-3f1e-40f3-9aab-db487d3bdd77 None None] MFA Rules not processed for user `fb0cf7d41be641a493d8b56e7d2eaf40`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:16.544731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-916c6c18-3f1e-40f3-9aab-db487d3bdd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:16.617479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3399131144f49ae97bb992f03c313a8', 'request_id': 'req-303e858a-30cd-4780-a746-b55717a475a4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:16.630638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3399131144f49ae97bb992f03c313a8', 'request_id': 'req-303e858a-30cd-4780-a746-b55717a475a4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:16.649171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3399131144f49ae97bb992f03c313a8', 'request_id': 'req-303e858a-30cd-4780-a746-b55717a475a4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:16.677830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 929/1853] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:16 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:16.754818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] The token cache is being invalidate because project e3399131144f49ae97bb992f03c313a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:16.755640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:16.755883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:16.757572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 925/1854] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:16 2022] DELETE /identity/v3/projects/e3399131144f49ae97bb992f03c313a8 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:16.762879 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b51ba9-0825-415f-926d-e11dacd4f7b3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:16.763172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b51ba9-0825-415f-926d-e11dacd4f7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:16.763486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b51ba9-0825-415f-926d-e11dacd4f7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:16.784729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d4b51ba9-0825-415f-926d-e11dacd4f7b3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:16.788113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d4b51ba9-0825-415f-926d-e11dacd4f7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:16.884015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 930/1855] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:16.891981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:16.891981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:16.895596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3298-ee25-4088-b53c-0f63fa991b7a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:16.904914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3298-ee25-4088-b53c-0f63fa991b7a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:16.904914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3298-ee25-4088-b53c-0f63fa991b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:16.915182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e858a-30cd-4780-a746-b55717a475a4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:16.920286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8c0b3298-ee25-4088-b53c-0f63fa991b7a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:16.922621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8c0b3298-ee25-4088-b53c-0f63fa991b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:16.934557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:16.935584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:16.935876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:16.937092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] PATH_INFO: `/v3/users/39896f5b78ab46b8a499ed4acbfacc96` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:16.939825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39896f5b78ab46b8a499ed4acbfacc96)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:16.944230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:16.950775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 931/1856] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:16.953876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b3298-ee25-4088-b53c-0f63fa991b7a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:16.979137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b3298-ee25-4088-b53c-0f63fa991b7a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:17.000943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:17.001713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.001985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.002289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.003109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:17.003494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:17.008114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:17.023266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 932/1857] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:17.027368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:17.047501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163534-00d7-428c-aa3f-ba9dc00a7f6f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:17.053417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ed6e-560c-4d02-8a07-26824b417f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:17.054372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ed6e-560c-4d02-8a07-26824b417f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.054763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ed6e-560c-4d02-8a07-26824b417f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.055151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ed6e-560c-4d02-8a07-26824b417f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.056082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792ed6e-560c-4d02-8a07-26824b417f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:17.058900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792ed6e-560c-4d02-8a07-26824b417f00 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:17.259463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 933/1858] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:17 2022] POST /identity/v3/projects => generated 373 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:17.261363 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ed6e-560c-4d02-8a07-26824b417f00 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:17.333675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39896f5b78ab46b8a499ed4acbfacc96'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:17.334878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39896f5b78ab46b8a499ed4acbfacc96'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:17.393061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ed6e-560c-4d02-8a07-26824b417f00 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:17.399487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:17.401051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.401051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.401555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.401922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:17.404899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:17.443271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39896f5b78ab46b8a499ed4acbfacc96'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:17.458029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39896f5b78ab46b8a499ed4acbfacc96'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:17.460202 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e386a919-2faf-49df-913f-a8580c2aa577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39896f5b78ab46b8a499ed4acbfacc96'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:17.492904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 926/1859] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:16 2022] DELETE /identity/v3/users/39896f5b78ab46b8a499ed4acbfacc96 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:17.500339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fe006-6c07-4ede-87a5-c7a80b9b4cf4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.500846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fe006-6c07-4ede-87a5-c7a80b9b4cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.501339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fe006-6c07-4ede-87a5-c7a80b9b4cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.527299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-396fe006-6c07-4ede-87a5-c7a80b9b4cf4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:17.528994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-396fe006-6c07-4ede-87a5-c7a80b9b4cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:17.547809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 934/1860] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:17 2022] POST /identity/v3/users => generated 287 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:17.553025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:17.555886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:17.572766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bc1a1-44d0-441b-b532-88fb1b8dc625 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:17.644821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 927/1861] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:17.659945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fe006-6c07-4ede-87a5-c7a80b9b4cf4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:17.661530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-396fe006-6c07-4ede-87a5-c7a80b9b4cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:17.667711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf8682-0bd8-40b5-9747-d105660829e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:17.668825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf8682-0bd8-40b5-9747-d105660829e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.669467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf8682-0bd8-40b5-9747-d105660829e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.669871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf8682-0bd8-40b5-9747-d105660829e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.670724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf8682-0bd8-40b5-9747-d105660829e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:17.672790 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf8682-0bd8-40b5-9747-d105660829e6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:17.684978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 935/1862] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:17.690797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf8682-0bd8-40b5-9747-d105660829e6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:17.705778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf8682-0bd8-40b5-9747-d105660829e6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:17.713578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bab6-0a97-41c4-b163-7e45f17be9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:17.714641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bab6-0a97-41c4-b163-7e45f17be9dc admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.715104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bab6-0a97-41c4-b163-7e45f17be9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.715623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bab6-0a97-41c4-b163-7e45f17be9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.716998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3bab6-0a97-41c4-b163-7e45f17be9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:17.720488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3bab6-0a97-41c4-b163-7e45f17be9dc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:17.735028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 936/1863] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:17.739834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bab6-0a97-41c4-b163-7e45f17be9dc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:17.740362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fe006-6c07-4ede-87a5-c7a80b9b4cf4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:17.744427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adff2d-a23b-42d5-89ae-88eca8ab6971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:17.745201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adff2d-a23b-42d5-89ae-88eca8ab6971 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.745484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adff2d-a23b-42d5-89ae-88eca8ab6971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.745766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adff2d-a23b-42d5-89ae-88eca8ab6971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.751483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bab6-0a97-41c4-b163-7e45f17be9dc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:17.757053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45657d7e-3558-4e91-874e-32c54f01ee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:17.757333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adff2d-a23b-42d5-89ae-88eca8ab6971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:17.758080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45657d7e-3558-4e91-874e-32c54f01ee29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.758357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45657d7e-3558-4e91-874e-32c54f01ee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.758614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adff2d-a23b-42d5-89ae-88eca8ab6971 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:17.758743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45657d7e-3558-4e91-874e-32c54f01ee29 admin admin] PATH_INFO: `/v3/projects/cc72d2e810404f6db19c5b23875603a3/users/eb90be224f384b79aea192d8d72c28de/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.770820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 928/1864] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:17.779087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45657d7e-3558-4e91-874e-32c54f01ee29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc72d2e810404f6db19c5b23875603a3, user_id=eb90be224f384b79aea192d8d72c28de, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:17.782445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45657d7e-3558-4e91-874e-32c54f01ee29 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:17.924222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56329-beac-40d7-80ac-fa7e0a92b486 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.924872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56329-beac-40d7-80ac-fa7e0a92b486 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.925478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56329-beac-40d7-80ac-fa7e0a92b486 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.927329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 929/1865] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:05:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 04:05:17.928935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 937/1866] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:17 2022] PUT /identity/v3/projects/cc72d2e810404f6db19c5b23875603a3/users/eb90be224f384b79aea192d8d72c28de/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:17.940252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c1f26-988e-4bfe-842e-3988447a126f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.940741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c1f26-988e-4bfe-842e-3988447a126f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.941296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c1f26-988e-4bfe-842e-3988447a126f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.944063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bb48b-dd6c-4be1-85b2-bc7baadb78ae None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:17.945037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bb48b-dd6c-4be1-85b2-bc7baadb78ae None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:17.946658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bb48b-dd6c-4be1-85b2-bc7baadb78ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:17.986600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-824bb48b-dd6c-4be1-85b2-bc7baadb78ae None None] MFA Rules not processed for user `eb90be224f384b79aea192d8d72c28de`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:17.988448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-824bb48b-dd6c-4be1-85b2-bc7baadb78ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:17.989878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a47c1f26-988e-4bfe-842e-3988447a126f None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:17.992495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a47c1f26-988e-4bfe-842e-3988447a126f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:18.079718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 938/1867] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:18.094416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 930/1868] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:05:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:18.098098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bb48b-dd6c-4be1-85b2-bc7baadb78ae None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:18.100652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-824bb48b-dd6c-4be1-85b2-bc7baadb78ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:18.111735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c1f26-988e-4bfe-842e-3988447a126f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:18.113585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a47c1f26-988e-4bfe-842e-3988447a126f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:18.130094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c1f26-988e-4bfe-842e-3988447a126f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:18.139799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5718-ea51-4aff-a22c-b22619c3b199 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:18.140490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5718-ea51-4aff-a22c-b22619c3b199 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:18.141592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5718-ea51-4aff-a22c-b22619c3b199 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:18.141883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5718-ea51-4aff-a22c-b22619c3b199 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:18.154349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5718-ea51-4aff-a22c-b22619c3b199 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:18.156439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5718-ea51-4aff-a22c-b22619c3b199 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:18.181463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 931/1869] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:18.220917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bb48b-dd6c-4be1-85b2-bc7baadb78ae None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:18.227279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:18.228395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:18.229277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:18.229749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:18.272979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:18.276206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:18.315812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 939/1870] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:18.348287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5718-ea51-4aff-a22c-b22619c3b199 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:18.466593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5718-ea51-4aff-a22c-b22619c3b199 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:18.473115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:18.474243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:18.474588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:18.474946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] PATH_INFO: `/v3/users/eb90be224f384b79aea192d8d72c28de` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:18.476373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb90be224f384b79aea192d8d72c28de)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:18.478483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:18.633323 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:18.635430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:18.655998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd3ee8-664b-4cdb-9eff-7ef5a01d0323 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:18.663785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:18.668529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:18.668529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:18.668529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] PATH_INFO: `/v3/projects/bfd70e00125c4f589c84ead04fb99cec` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:18.668529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd70e00125c4f589c84ead04fb99cec)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:18.669046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:18.774764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb90be224f384b79aea192d8d72c28de'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:18.775761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb90be224f384b79aea192d8d72c28de'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:18.857221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd70e00125c4f589c84ead04fb99cec', 'request_id': 'req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:18.869086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd70e00125c4f589c84ead04fb99cec', 'request_id': 'req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:18.899806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb90be224f384b79aea192d8d72c28de'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:18.909615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb90be224f384b79aea192d8d72c28de'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:18.910442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-14bf4f4b-7841-4784-911e-b384d29847f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb90be224f384b79aea192d8d72c28de'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:18.933391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 932/1871] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:18 2022] DELETE /identity/v3/users/eb90be224f384b79aea192d8d72c28de => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:18.938730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2d3c1-e16a-48b5-bba5-355d59dc07b7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:18.939161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2d3c1-e16a-48b5-bba5-355d59dc07b7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:18.940089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2d3c1-e16a-48b5-bba5-355d59dc07b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:18.961083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-27b2d3c1-e16a-48b5-bba5-355d59dc07b7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:18.970758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-27b2d3c1-e16a-48b5-bba5-355d59dc07b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:19.035798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 933/1872] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:19.046755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2d3c1-e16a-48b5-bba5-355d59dc07b7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:19.049062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-27b2d3c1-e16a-48b5-bba5-355d59dc07b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:19.122915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd70e00125c4f589c84ead04fb99cec', 'request_id': 'req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.126375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2d3c1-e16a-48b5-bba5-355d59dc07b7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:19.132372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:19.133104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:19.133415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:19.138955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:19.146909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:19.148566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:19.155338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] The token cache is being invalidate because project bfd70e00125c4f589c84ead04fb99cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:19.156223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.158434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.161942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 940/1873] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:18 2022] DELETE /identity/v3/projects/bfd70e00125c4f589c84ead04fb99cec => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:19.168314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:19.170571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:19.171600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 934/1874] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:19.204553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:19.259201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2284b94-1c2c-40ad-a4bf-e5873ac4ab16 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:19.265362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:19.266908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:19.267341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:19.267684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] PATH_INFO: `/v3/users/db1b22b6bcd2452a8effb2dc47369451` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:19.269305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1b22b6bcd2452a8effb2dc47369451)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:19.271957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:19.350053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:19.353221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:19.382128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80bcc7-182d-40f0-a8f8-ffbed9779df1 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:19.397472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:19.399923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:19.404858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:19.404858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] PATH_INFO: `/v3/users/745098eb1d8f48bdb025de871a3f7202` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:19.404858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=745098eb1d8f48bdb025de871a3f7202)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:19.408357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:19.501341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:19.656889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1b22b6bcd2452a8effb2dc47369451'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.660042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1b22b6bcd2452a8effb2dc47369451'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.662045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1b22b6bcd2452a8effb2dc47369451'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.739413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '745098eb1d8f48bdb025de871a3f7202'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.741058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '745098eb1d8f48bdb025de871a3f7202'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.755793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1b22b6bcd2452a8effb2dc47369451'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.772805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1b22b6bcd2452a8effb2dc47369451'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.921697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '745098eb1d8f48bdb025de871a3f7202'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.933713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '745098eb1d8f48bdb025de871a3f7202'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.934642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '745098eb1d8f48bdb025de871a3f7202'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:19.991634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 935/1875] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:19 2022] DELETE /identity/v3/users/745098eb1d8f48bdb025de871a3f7202 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:20.038495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 941/1876] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:19 2022] DELETE /identity/v3/users/db1b22b6bcd2452a8effb2dc47369451 => generated 0 bytes in 867 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:20.379536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:20.384736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:20.532801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef1234-a85b-4d9d-8e67-27620937de1d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:20.599184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:20.601670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:20.646757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:20.648370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:20.648906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:20.649489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] PATH_INFO: `/v3/projects/cc72d2e810404f6db19c5b23875603a3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:20.654271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc72d2e810404f6db19c5b23875603a3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:20.659304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:20.775842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536fcb5-fec9-419f-88df-0dd5fd3bb471 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:20.787790 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:20.790781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:20.791353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:20.791925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] PATH_INFO: `/v3/projects/27bab7586bb548ff935b2ec2ed9acc06` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:20.795862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bab7586bb548ff935b2ec2ed9acc06)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:20.801991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:20.946877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc72d2e810404f6db19c5b23875603a3', 'request_id': 'req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:20.956194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc72d2e810404f6db19c5b23875603a3', 'request_id': 'req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:20.967298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc72d2e810404f6db19c5b23875603a3', 'request_id': 'req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:20.998122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bab7586bb548ff935b2ec2ed9acc06', 'request_id': 'req-3a14dba9-4e73-4279-aec3-03b0c16ba044'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.007482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bab7586bb548ff935b2ec2ed9acc06', 'request_id': 'req-3a14dba9-4e73-4279-aec3-03b0c16ba044'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.158763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] The token cache is being invalidate because project cc72d2e810404f6db19c5b23875603a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:21.159074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.162288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.169586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 936/1877] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:20 2022] DELETE /identity/v3/projects/cc72d2e810404f6db19c5b23875603a3 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:21.170552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:21.175257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:21.196988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bab7586bb548ff935b2ec2ed9acc06', 'request_id': 'req-3a14dba9-4e73-4279-aec3-03b0c16ba044'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.215591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] The token cache is being invalidate because project 27bab7586bb548ff935b2ec2ed9acc06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:21.215885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.217286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3a14dba9-4e73-4279-aec3-03b0c16ba044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.221322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 942/1878] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:20 2022] DELETE /identity/v3/projects/27bab7586bb548ff935b2ec2ed9acc06 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:21.224116 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f168-464b-4dde-b8fa-e11e7e1e93e6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:21.224867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f168-464b-4dde-b8fa-e11e7e1e93e6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:21.225305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f168-464b-4dde-b8fa-e11e7e1e93e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:21.258526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ca91f168-464b-4dde-b8fa-e11e7e1e93e6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:21.258526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ca91f168-464b-4dde-b8fa-e11e7e1e93e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:21.273823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b61f6-5f86-40f7-8d84-6e9a298d959c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:21.331994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:21.348831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 943/1879] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:21.350139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f168-464b-4dde-b8fa-e11e7e1e93e6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:21.353018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ca91f168-464b-4dde-b8fa-e11e7e1e93e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:21.378917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f168-464b-4dde-b8fa-e11e7e1e93e6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:21.390026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:21.391946 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:21.392627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:21.393114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] PATH_INFO: `/v3/users/fb0cf7d41be641a493d8b56e7d2eaf40` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:21.399664 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0cf7d41be641a493d8b56e7d2eaf40)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:21.411822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:21.421280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:21.424610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:21.425041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:21.425455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] PATH_INFO: `/v3/projects/1c2334fa740b46e0b9e20d5b9192501b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:21.428476 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2334fa740b46e0b9e20d5b9192501b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:21.431844 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:21.518808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:21.753667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2334fa740b46e0b9e20d5b9192501b', 'request_id': 'req-3d7decff-a5bd-4a88-b359-49653c8389d1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.763024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2334fa740b46e0b9e20d5b9192501b', 'request_id': 'req-3d7decff-a5bd-4a88-b359-49653c8389d1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.770922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2334fa740b46e0b9e20d5b9192501b', 'request_id': 'req-3d7decff-a5bd-4a88-b359-49653c8389d1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.795589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0cf7d41be641a493d8b56e7d2eaf40'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.797896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0cf7d41be641a493d8b56e7d2eaf40'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.798681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0cf7d41be641a493d8b56e7d2eaf40'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.819353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0cf7d41be641a493d8b56e7d2eaf40'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.833377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ecdffa4-a1d6-4fe9-b0e1-1dfc28157a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0cf7d41be641a493d8b56e7d2eaf40'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.920737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] The token cache is being invalidate because project 1c2334fa740b46e0b9e20d5b9192501b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:21.921594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.923405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3d7decff-a5bd-4a88-b359-49653c8389d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:21.926512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 937/1880] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:21 2022] DELETE /identity/v3/projects/1c2334fa740b46e0b9e20d5b9192501b => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:21.928522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf40248-7215-46f2-a641-047e318b851c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:21.928982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf40248-7215-46f2-a641-047e318b851c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:21.929394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf40248-7215-46f2-a641-047e318b851c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:21.966848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ccf40248-7215-46f2-a641-047e318b851c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:21.969340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ccf40248-7215-46f2-a641-047e318b851c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:22.068753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 938/1881] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:22.071616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cf5c1-9a81-490d-8769-8d2983c12033 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.072398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cf5c1-9a81-490d-8769-8d2983c12033 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.073355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cf5c1-9a81-490d-8769-8d2983c12033 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.076162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 939/1882] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:05:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:05:22.077928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd37d6b-6ce3-47af-872a-365d496069cb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.081291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd37d6b-6ce3-47af-872a-365d496069cb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.081291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd37d6b-6ce3-47af-872a-365d496069cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.103344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3cd37d6b-6ce3-47af-872a-365d496069cb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:22.110193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3cd37d6b-6ce3-47af-872a-365d496069cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:22.130067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 940/1883] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:22.130324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd37d6b-6ce3-47af-872a-365d496069cb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:22.157518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd37d6b-6ce3-47af-872a-365d496069cb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:22.166106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8388a1-cfe8-4870-a9a8-e0dcad3916d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:22.167745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8388a1-cfe8-4870-a9a8-e0dcad3916d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.168386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8388a1-cfe8-4870-a9a8-e0dcad3916d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.169066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8388a1-cfe8-4870-a9a8-e0dcad3916d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.170642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8388a1-cfe8-4870-a9a8-e0dcad3916d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:22.171349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8388a1-cfe8-4870-a9a8-e0dcad3916d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:22.175509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8388a1-cfe8-4870-a9a8-e0dcad3916d6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:22.207904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 944/1884] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:21 2022] DELETE /identity/v3/users/fb0cf7d41be641a493d8b56e7d2eaf40 => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:22.214216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544372ca-43a5-4452-b326-efc47edcdcd9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.214883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544372ca-43a5-4452-b326-efc47edcdcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.215544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544372ca-43a5-4452-b326-efc47edcdcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.247812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 941/1885] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:22.253643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6a57c-2670-4ceb-baf2-0a8862279a6a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.253643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6a57c-2670-4ceb-baf2-0a8862279a6a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.253643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6a57c-2670-4ceb-baf2-0a8862279a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.267657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-544372ca-43a5-4452-b326-efc47edcdcd9 None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:22.271279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-544372ca-43a5-4452-b326-efc47edcdcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:22.309353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-67d6a57c-2670-4ceb-baf2-0a8862279a6a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:22.313129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-67d6a57c-2670-4ceb-baf2-0a8862279a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:22.507988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 945/1886] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:05:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:22.524110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544372ca-43a5-4452-b326-efc47edcdcd9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:22.564264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 942/1887] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:22.574329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6a57c-2670-4ceb-baf2-0a8862279a6a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:22.609500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6a57c-2670-4ceb-baf2-0a8862279a6a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:22.630310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:22.631547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.631984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.632473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.633895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:22.634462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:22.639047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:22.671322 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:22.684506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 943/1888] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:22.686766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:22.697814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:22.755132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cba49-51ff-462b-a0b6-b44e9799d4b6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:22.764685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423fa4f-1584-451d-8ca5-88ea17ae5087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:22.767007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423fa4f-1584-451d-8ca5-88ea17ae5087 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.767633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423fa4f-1584-451d-8ca5-88ea17ae5087 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.768779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423fa4f-1584-451d-8ca5-88ea17ae5087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.784446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423fa4f-1584-451d-8ca5-88ea17ae5087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:22.787123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423fa4f-1584-451d-8ca5-88ea17ae5087 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:22.816361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 944/1889] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:22.820244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423fa4f-1584-451d-8ca5-88ea17ae5087 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:22.838841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544372ca-43a5-4452-b326-efc47edcdcd9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:22.849615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:22.851021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.851466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.852219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.853674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:22.858335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:22.867509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423fa4f-1584-451d-8ca5-88ea17ae5087 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:22.884468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00802cb7-3d8e-4af8-a91d-541ed62e7e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:22.885693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00802cb7-3d8e-4af8-a91d-541ed62e7e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:22.886181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00802cb7-3d8e-4af8-a91d-541ed62e7e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:22.886662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00802cb7-3d8e-4af8-a91d-541ed62e7e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:22.887951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00802cb7-3d8e-4af8-a91d-541ed62e7e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:22.894826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00802cb7-3d8e-4af8-a91d-541ed62e7e26 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.031548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 946/1890] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:22 2022] POST /identity/v3/projects => generated 377 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:23.037111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.040859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:23.173255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62c4e8-5f15-44d7-859f-406775c3f791 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.179619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.189136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.189440 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.189578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] PATH_INFO: `/v3/projects/cbdb6ffadf8144f5bf9319df3219d0bd` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.191024 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbdb6ffadf8144f5bf9319df3219d0bd)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.193078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.224816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 945/1891] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:22 2022] POST /identity/v3/projects => generated 399 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:23.226408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00802cb7-3d8e-4af8-a91d-541ed62e7e26 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.331623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00802cb7-3d8e-4af8-a91d-541ed62e7e26 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.337719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab9f7-3ef5-483b-8911-ed53292f3c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.338523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab9f7-3ef5-483b-8911-ed53292f3c5e admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.338878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab9f7-3ef5-483b-8911-ed53292f3c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.342826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab9f7-3ef5-483b-8911-ed53292f3c5e admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.342826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bab9f7-3ef5-483b-8911-ed53292f3c5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.342826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bab9f7-3ef5-483b-8911-ed53292f3c5e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.387151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbdb6ffadf8144f5bf9319df3219d0bd', 'request_id': 'req-1cf7b695-2ca2-4016-a001-cda7e4545b9e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:23.395075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbdb6ffadf8144f5bf9319df3219d0bd', 'request_id': 'req-1cf7b695-2ca2-4016-a001-cda7e4545b9e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:23.480051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 946/1892] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:23 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:23.481589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab9f7-3ef5-483b-8911-ed53292f3c5e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.496095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab9f7-3ef5-483b-8911-ed53292f3c5e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.503509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:23.505228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbdb6ffadf8144f5bf9319df3219d0bd', 'request_id': 'req-1cf7b695-2ca2-4016-a001-cda7e4545b9e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:23.519168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] The token cache is being invalidate because project cbdb6ffadf8144f5bf9319df3219d0bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:23.519597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:23.520428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:23.525773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 947/1893] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:23 2022] DELETE /identity/v3/projects/cbdb6ffadf8144f5bf9319df3219d0bd => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:23.526730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.529497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:23.586522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3760f-8fb0-4ab2-9d90-cea26cf108e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.587835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3760f-8fb0-4ab2-9d90-cea26cf108e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.588324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3760f-8fb0-4ab2-9d90-cea26cf108e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.588809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3760f-8fb0-4ab2-9d90-cea26cf108e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.590911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3760f-8fb0-4ab2-9d90-cea26cf108e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.596459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3760f-8fb0-4ab2-9d90-cea26cf108e1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.627679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7b695-2ca2-4016-a001-cda7e4545b9e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.631595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e59792-ced8-4aff-a847-3935b8e13bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.632309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e59792-ced8-4aff-a847-3935b8e13bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.632890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e59792-ced8-4aff-a847-3935b8e13bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.633180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e59792-ced8-4aff-a847-3935b8e13bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.633977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e59792-ced8-4aff-a847-3935b8e13bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.635777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e59792-ced8-4aff-a847-3935b8e13bcb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.654705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 948/1894] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:23.657748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922e4a-3ba5-4314-ab2f-b3a93831e490 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.658204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922e4a-3ba5-4314-ab2f-b3a93831e490 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.658673 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922e4a-3ba5-4314-ab2f-b3a93831e490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.685571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fc922e4a-3ba5-4314-ab2f-b3a93831e490 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:23.687161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fc922e4a-3ba5-4314-ab2f-b3a93831e490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:23.703406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 949/1895] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:23.703406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc922e4a-3ba5-4314-ab2f-b3a93831e490 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.712962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc922e4a-3ba5-4314-ab2f-b3a93831e490 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.719030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f583973-2925-4962-8d17-63323fdf587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.720227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f583973-2925-4962-8d17-63323fdf587f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.720867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f583973-2925-4962-8d17-63323fdf587f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.721221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f583973-2925-4962-8d17-63323fdf587f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.722093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f583973-2925-4962-8d17-63323fdf587f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.724995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f583973-2925-4962-8d17-63323fdf587f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.741698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 950/1896] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:23.742911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48dba5-0c72-4e9d-aa0c-4ddf333c6942 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.743208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48dba5-0c72-4e9d-aa0c-4ddf333c6942 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.743738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48dba5-0c72-4e9d-aa0c-4ddf333c6942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.745348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 947/1897] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:23 2022] POST /identity/v3/users => generated 300 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:23.751450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3760f-8fb0-4ab2-9d90-cea26cf108e1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.765246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3760f-8fb0-4ab2-9d90-cea26cf108e1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.775033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8ddf-9bb9-481e-b700-da8c65d6eb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.775993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-db48dba5-0c72-4e9d-aa0c-4ddf333c6942 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:23.776456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8ddf-9bb9-481e-b700-da8c65d6eb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.776884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8ddf-9bb9-481e-b700-da8c65d6eb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.777275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8ddf-9bb9-481e-b700-da8c65d6eb8a admin admin] PATH_INFO: `/v3/projects/68327a104e964ddca1304b29661f83e6/users/430ca4a6af054228802290aa876120eb/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.777540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-db48dba5-0c72-4e9d-aa0c-4ddf333c6942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:23.788538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 951/1898] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:23.789588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48dba5-0c72-4e9d-aa0c-4ddf333c6942 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.791133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-db48dba5-0c72-4e9d-aa0c-4ddf333c6942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:23.803385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8ddf-9bb9-481e-b700-da8c65d6eb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=68327a104e964ddca1304b29661f83e6, user_id=430ca4a6af054228802290aa876120eb, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.804766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48dba5-0c72-4e9d-aa0c-4ddf333c6942 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.807196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8ddf-9bb9-481e-b700-da8c65d6eb8a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.811619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21072dd8-4f49-4565-b21c-6830686ded00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.812782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072dd8-4f49-4565-b21c-6830686ded00 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.813146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072dd8-4f49-4565-b21c-6830686ded00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.813504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072dd8-4f49-4565-b21c-6830686ded00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.814344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21072dd8-4f49-4565-b21c-6830686ded00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.816095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21072dd8-4f49-4565-b21c-6830686ded00 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.830261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 952/1899] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:23.837502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21072dd8-4f49-4565-b21c-6830686ded00 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.845903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21072dd8-4f49-4565-b21c-6830686ded00 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.852403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.853407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.853798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.854185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.855111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:23.855638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.857891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.874550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 953/1900] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:23.875734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.891600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74659713-4594-4dd4-b04c-5ec7b76f0d89 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.901784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9a913-e972-41fa-b7e2-4b69cbae7371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.903431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a913-e972-41fa-b7e2-4b69cbae7371 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.904101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a913-e972-41fa-b7e2-4b69cbae7371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.904680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a913-e972-41fa-b7e2-4b69cbae7371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.906652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9a913-e972-41fa-b7e2-4b69cbae7371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.910147 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9a913-e972-41fa-b7e2-4b69cbae7371 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.928568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 954/1901] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:23.929522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9a913-e972-41fa-b7e2-4b69cbae7371 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:23.941988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9a913-e972-41fa-b7e2-4b69cbae7371 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:23.947414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62632f-d96d-4c1d-a602-0a9420906701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:23.948104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62632f-d96d-4c1d-a602-0a9420906701 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:23.955509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62632f-d96d-4c1d-a602-0a9420906701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:23.955509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62632f-d96d-4c1d-a602-0a9420906701 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:23.955509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62632f-d96d-4c1d-a602-0a9420906701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:23.955509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62632f-d96d-4c1d-a602-0a9420906701 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:23.988095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 948/1902] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:23 2022] PUT /identity/v3/projects/68327a104e964ddca1304b29661f83e6/users/430ca4a6af054228802290aa876120eb/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:23.989849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8ddf-9bb9-481e-b700-da8c65d6eb8a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:24.083757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8ddf-9bb9-481e-b700-da8c65d6eb8a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:24.090195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414bf1d-0417-48bb-8100-a427ecae8572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:24.091930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414bf1d-0417-48bb-8100-a427ecae8572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.092332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414bf1d-0417-48bb-8100-a427ecae8572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.092813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414bf1d-0417-48bb-8100-a427ecae8572 admin admin] PATH_INFO: `/v3/projects/292f393c00b24047b4ac8620034ea64b/users/01c53532edd34321ac16438229bfd3b0/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.119664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414bf1d-0417-48bb-8100-a427ecae8572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=292f393c00b24047b4ac8620034ea64b, user_id=01c53532edd34321ac16438229bfd3b0, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:24.127820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414bf1d-0417-48bb-8100-a427ecae8572 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:24.177793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 955/1903] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:23 2022] POST /identity/v3/projects => generated 365 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:24.180535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c6107-40b7-42f4-916f-f43e9ba1abe3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.181156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c6107-40b7-42f4-916f-f43e9ba1abe3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.181721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c6107-40b7-42f4-916f-f43e9ba1abe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.225828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8f3c6107-40b7-42f4-916f-f43e9ba1abe3 None None] MFA Rules not processed for user `430ca4a6af054228802290aa876120eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:24.227266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8f3c6107-40b7-42f4-916f-f43e9ba1abe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:24.311608 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 956/1904] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:24.325719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 949/1905] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:23 2022] PUT /identity/v3/projects/292f393c00b24047b4ac8620034ea64b/users/01c53532edd34321ac16438229bfd3b0/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:24.325719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414bf1d-0417-48bb-8100-a427ecae8572 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:24.331870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254899c-8da6-4623-b2f7-bdb2a1a7fcfc None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.333284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254899c-8da6-4623-b2f7-bdb2a1a7fcfc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.333284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254899c-8da6-4623-b2f7-bdb2a1a7fcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.379684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3254899c-8da6-4623-b2f7-bdb2a1a7fcfc None None] MFA Rules not processed for user `01c53532edd34321ac16438229bfd3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:24.381816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3254899c-8da6-4623-b2f7-bdb2a1a7fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:24.430662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414bf1d-0417-48bb-8100-a427ecae8572 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:24.442262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ff602-507d-4bd6-85ef-ecf6c4f2f3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:24.443732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ff602-507d-4bd6-85ef-ecf6c4f2f3f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.444323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ff602-507d-4bd6-85ef-ecf6c4f2f3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.444909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ff602-507d-4bd6-85ef-ecf6c4f2f3f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.453998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ff602-507d-4bd6-85ef-ecf6c4f2f3f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:24.453998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ff602-507d-4bd6-85ef-ecf6c4f2f3f2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:24.525167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 957/1906] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:24.526804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254899c-8da6-4623-b2f7-bdb2a1a7fcfc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:24.531971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3254899c-8da6-4623-b2f7-bdb2a1a7fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:24.602687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254899c-8da6-4623-b2f7-bdb2a1a7fcfc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:24.606782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc914b-0b48-476f-bf87-153886e4a3b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8327d9916424843b2a67cbdd856ad37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:24.607584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc914b-0b48-476f-bf87-153886e4a3b3 service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.607878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc914b-0b48-476f-bf87-153886e4a3b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.608157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc914b-0b48-476f-bf87-153886e4a3b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.619886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 950/1907] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:24 2022] POST /identity/v3/users => generated 283 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:24.620829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ff602-507d-4bd6-85ef-ecf6c4f2f3f2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:24.632412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ff602-507d-4bd6-85ef-ecf6c4f2f3f2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:24.637907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc40ec-1159-49fc-9c79-1007f4a1c505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:24.638629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc40ec-1159-49fc-9c79-1007f4a1c505 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.638906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc40ec-1159-49fc-9c79-1007f4a1c505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.639187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc40ec-1159-49fc-9c79-1007f4a1c505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.639939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc40ec-1159-49fc-9c79-1007f4a1c505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:24.643481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc40ec-1159-49fc-9c79-1007f4a1c505 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:24.717682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc914b-0b48-476f-bf87-153886e4a3b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:24.720217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc914b-0b48-476f-bf87-153886e4a3b3 service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:24.732799 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 958/1908] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:24.734445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc914b-0b48-476f-bf87-153886e4a3b3 service glance] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:24.767586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc914b-0b48-476f-bf87-153886e4a3b3 service glance] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:24.768943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677564f4-8d07-456f-bc16-5c9d52646df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:24.770020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677564f4-8d07-456f-bc16-5c9d52646df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.773401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677564f4-8d07-456f-bc16-5c9d52646df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.773401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677564f4-8d07-456f-bc16-5c9d52646df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.773401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677564f4-8d07-456f-bc16-5c9d52646df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:24.775862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677564f4-8d07-456f-bc16-5c9d52646df9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:24.790006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 951/1909] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:24 2022] POST /identity/v3/projects => generated 401 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:24.791797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc40ec-1159-49fc-9c79-1007f4a1c505 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:24.797728 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 959/1910] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:24.803578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677564f4-8d07-456f-bc16-5c9d52646df9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:24.823854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677564f4-8d07-456f-bc16-5c9d52646df9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:24.931717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc40ec-1159-49fc-9c79-1007f4a1c505 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:24.941653 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2776bc-c393-498b-89a7-9bfe3aa418a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:24.942771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2776bc-c393-498b-89a7-9bfe3aa418a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.943265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2776bc-c393-498b-89a7-9bfe3aa418a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.943764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2776bc-c393-498b-89a7-9bfe3aa418a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.944911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2776bc-c393-498b-89a7-9bfe3aa418a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:24.948187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2776bc-c393-498b-89a7-9bfe3aa418a8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:24.978721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a640e-4cec-4d46-81a8-103022873fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:24.980111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a640e-4cec-4d46-81a8-103022873fee admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:24.980344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a640e-4cec-4d46-81a8-103022873fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:24.980843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a640e-4cec-4d46-81a8-103022873fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:24.984585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a640e-4cec-4d46-81a8-103022873fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:24.995356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a640e-4cec-4d46-81a8-103022873fee admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:25.042862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 960/1911] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:25.048232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a640e-4cec-4d46-81a8-103022873fee admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:25.067760 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a640e-4cec-4d46-81a8-103022873fee admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:25.079171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a73c1-0823-45be-8fe7-ac0fc6baf56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:25.083812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a73c1-0823-45be-8fe7-ac0fc6baf56a admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:25.085386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a73c1-0823-45be-8fe7-ac0fc6baf56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:25.086856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a73c1-0823-45be-8fe7-ac0fc6baf56a admin admin] PATH_INFO: `/v3/projects/f063bf26a8ce42eebdc5faa690c55a49/users/7cfaeed15e3f480bab4238b63f38ead4/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:25.125412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a73c1-0823-45be-8fe7-ac0fc6baf56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f063bf26a8ce42eebdc5faa690c55a49, user_id=7cfaeed15e3f480bab4238b63f38ead4, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:25.131554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a73c1-0823-45be-8fe7-ac0fc6baf56a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:25.151547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 952/1912] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:24 2022] POST /identity/v3/users => generated 307 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:25.164139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2776bc-c393-498b-89a7-9bfe3aa418a8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:25.192867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2776bc-c393-498b-89a7-9bfe3aa418a8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:25.207825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f0736-afb9-4c17-bccd-f3944f9ca415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:25.209100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f0736-afb9-4c17-bccd-f3944f9ca415 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:25.212772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f0736-afb9-4c17-bccd-f3944f9ca415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:25.212772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f0736-afb9-4c17-bccd-f3944f9ca415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:25.212772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f0736-afb9-4c17-bccd-f3944f9ca415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:25.222434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f0736-afb9-4c17-bccd-f3944f9ca415 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:25.258663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 961/1913] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:25 2022] PUT /identity/v3/projects/f063bf26a8ce42eebdc5faa690c55a49/users/7cfaeed15e3f480bab4238b63f38ead4/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:25.275317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a73c1-0823-45be-8fe7-ac0fc6baf56a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:25.276645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 953/1914] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:25.284741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b12d0-73be-4d28-ad32-3b42a26d137e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:25.285202 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b12d0-73be-4d28-ad32-3b42a26d137e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:25.285651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b12d0-73be-4d28-ad32-3b42a26d137e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:25.317438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:25.352911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-255b12d0-73be-4d28-ad32-3b42a26d137e None None] MFA Rules not processed for user `7cfaeed15e3f480bab4238b63f38ead4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:25.357038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-255b12d0-73be-4d28-ad32-3b42a26d137e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:25.410750 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a73c1-0823-45be-8fe7-ac0fc6baf56a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:25.428073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3071c4-32f2-4461-ad66-99265ec76cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:25.430725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3071c4-32f2-4461-ad66-99265ec76cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:25.431159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3071c4-32f2-4461-ad66-99265ec76cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:25.431356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3071c4-32f2-4461-ad66-99265ec76cc6 admin admin] PATH_INFO: `/v3/projects/f9749cd41118426a892faa6178245141/users/ac343c4acdc74ceebdf876b4cf8deea8/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:25.455912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3071c4-32f2-4461-ad66-99265ec76cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9749cd41118426a892faa6178245141, user_id=ac343c4acdc74ceebdf876b4cf8deea8, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:25.459601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3071c4-32f2-4461-ad66-99265ec76cc6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:25.549284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 954/1915] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:25.551279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b12d0-73be-4d28-ad32-3b42a26d137e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:25.575247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 962/1916] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:25 2022] PUT /identity/v3/projects/f9749cd41118426a892faa6178245141/users/ac343c4acdc74ceebdf876b4cf8deea8/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:25.583521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3071c4-32f2-4461-ad66-99265ec76cc6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:25.619888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b12d0-73be-4d28-ad32-3b42a26d137e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:25.728982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13e6e3-f59c-4f90-9c46-a2c77495c925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:25.730097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e6e3-f59c-4f90-9c46-a2c77495c925 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:25.730634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e6e3-f59c-4f90-9c46-a2c77495c925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:25.731124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e6e3-f59c-4f90-9c46-a2c77495c925 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:25.732747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13e6e3-f59c-4f90-9c46-a2c77495c925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:25.737043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13e6e3-f59c-4f90-9c46-a2c77495c925 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:25.761574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3071c4-32f2-4461-ad66-99265ec76cc6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:25.775447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facf786-457a-4a8f-abaa-f5dae169dcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:25.776624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facf786-457a-4a8f-abaa-f5dae169dcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:25.779784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facf786-457a-4a8f-abaa-f5dae169dcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:25.783937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facf786-457a-4a8f-abaa-f5dae169dcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:25.783937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facf786-457a-4a8f-abaa-f5dae169dcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:25.796197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facf786-457a-4a8f-abaa-f5dae169dcd3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:25.817192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 963/1917] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:25.818285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facf786-457a-4a8f-abaa-f5dae169dcd3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:25.832403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facf786-457a-4a8f-abaa-f5dae169dcd3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:25.838643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1482a1-8417-4edf-be39-93598f4e746a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:25.839611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1482a1-8417-4edf-be39-93598f4e746a admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:25.840045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1482a1-8417-4edf-be39-93598f4e746a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:25.840609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1482a1-8417-4edf-be39-93598f4e746a admin admin] PATH_INFO: `/v3/projects/f9749cd41118426a892faa6178245141/users/ac343c4acdc74ceebdf876b4cf8deea8/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:25.843035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1482a1-8417-4edf-be39-93598f4e746a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9749cd41118426a892faa6178245141, user_id=ac343c4acdc74ceebdf876b4cf8deea8, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:25.846407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1482a1-8417-4edf-be39-93598f4e746a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:25.863108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 964/1918] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:25 2022] PUT /identity/v3/projects/f9749cd41118426a892faa6178245141/users/ac343c4acdc74ceebdf876b4cf8deea8/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:25.876757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c006d98-dc7c-4b17-a558-e71aa2282ed6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:25.877389 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c006d98-dc7c-4b17-a558-e71aa2282ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:25.877792 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c006d98-dc7c-4b17-a558-e71aa2282ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:25.903997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 955/1919] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:25 2022] POST /identity/v3/projects => generated 365 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:25.911560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13e6e3-f59c-4f90-9c46-a2c77495c925 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:25.939700 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4c006d98-dc7c-4b17-a558-e71aa2282ed6 None None] MFA Rules not processed for user `ac343c4acdc74ceebdf876b4cf8deea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:25.941081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4c006d98-dc7c-4b17-a558-e71aa2282ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:26.040256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13e6e3-f59c-4f90-9c46-a2c77495c925 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:26.047591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:26.048438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:26.048877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:26.049254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:26.050110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:26.054593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:26.073059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 965/1920] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:26.170490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 956/1921] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:25 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:05:26.179513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c006d98-dc7c-4b17-a558-e71aa2282ed6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:26.211839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c006d98-dc7c-4b17-a558-e71aa2282ed6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:26.213266 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:26.219888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:26.220114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145235da-f568-4567-aed9-b41b1231f7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:26.220839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145235da-f568-4567-aed9-b41b1231f7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:26.221751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145235da-f568-4567-aed9-b41b1231f7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:26.221751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145235da-f568-4567-aed9-b41b1231f7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:26.222937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145235da-f568-4567-aed9-b41b1231f7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:26.225354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145235da-f568-4567-aed9-b41b1231f7d0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:26.244467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 966/1922] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:26.250952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145235da-f568-4567-aed9-b41b1231f7d0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:26.269241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145235da-f568-4567-aed9-b41b1231f7d0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:26.277547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bb49d-5d93-4083-918e-55692af23096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:26.279141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bb49d-5d93-4083-918e-55692af23096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:26.279583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bb49d-5d93-4083-918e-55692af23096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:26.279926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bb49d-5d93-4083-918e-55692af23096 admin admin] PATH_INFO: `/v3/projects/40ee0ac154e14db186632adda367e833/users/f1e16c17cd0c4be690c693da833da8bd/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:26.318166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:26.346274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bb49d-5d93-4083-918e-55692af23096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ee0ac154e14db186632adda367e833, user_id=f1e16c17cd0c4be690c693da833da8bd, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:26.350019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bb49d-5d93-4083-918e-55692af23096 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:26.352275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972563-09a9-4d8f-ac01-0cbfd2380c32 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:26.378628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:26.380362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:26.380713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:26.381210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:26.420557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:26.421098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:26.445605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 957/1923] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:26 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:26.446630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:26.456190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:26.486481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 967/1924] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:26 2022] PUT /identity/v3/projects/40ee0ac154e14db186632adda367e833/users/f1e16c17cd0c4be690c693da833da8bd/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:26.487346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d6caf-8988-46f0-8224-81e06e8e1802 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:26.487839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bb49d-5d93-4083-918e-55692af23096 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:26.625511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:26.627009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:26.627530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:26.628053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] PATH_INFO: `/v3/users/a07facc432e744ddbaafba4134ec73df` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:26.630538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07facc432e744ddbaafba4134ec73df)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:26.634662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:26.645312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bb49d-5d93-4083-918e-55692af23096 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:26.684962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948c0e-6bc9-414a-a96b-bd4eaa74850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:26.688100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948c0e-6bc9-414a-a96b-bd4eaa74850a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:26.690857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948c0e-6bc9-414a-a96b-bd4eaa74850a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:26.691375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948c0e-6bc9-414a-a96b-bd4eaa74850a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:26.692333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d948c0e-6bc9-414a-a96b-bd4eaa74850a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:26.701761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d948c0e-6bc9-414a-a96b-bd4eaa74850a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:26.726778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 968/1925] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:26.732645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948c0e-6bc9-414a-a96b-bd4eaa74850a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:26.752593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948c0e-6bc9-414a-a96b-bd4eaa74850a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:26.758826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5ac31-3538-45f2-b10f-3b216b4f80a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:26.759765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5ac31-3538-45f2-b10f-3b216b4f80a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:26.760038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5ac31-3538-45f2-b10f-3b216b4f80a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:26.760413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5ac31-3538-45f2-b10f-3b216b4f80a6 admin admin] PATH_INFO: `/v3/projects/40ee0ac154e14db186632adda367e833/users/f1e16c17cd0c4be690c693da833da8bd/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:26.763472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5ac31-3538-45f2-b10f-3b216b4f80a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ee0ac154e14db186632adda367e833, user_id=f1e16c17cd0c4be690c693da833da8bd, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:26.766444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5ac31-3538-45f2-b10f-3b216b4f80a6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:26.801666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 969/1926] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:26 2022] PUT /identity/v3/projects/40ee0ac154e14db186632adda367e833/users/f1e16c17cd0c4be690c693da833da8bd/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:26.802593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5ac31-3538-45f2-b10f-3b216b4f80a6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:26.904416 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5ac31-3538-45f2-b10f-3b216b4f80a6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:26.916330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafad07-146c-46ba-92c6-bf49f325b5b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:26.916330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafad07-146c-46ba-92c6-bf49f325b5b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:26.916330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafad07-146c-46ba-92c6-bf49f325b5b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:26.916330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafad07-146c-46ba-92c6-bf49f325b5b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:27.007708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafad07-146c-46ba-92c6-bf49f325b5b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:27.011066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafad07-146c-46ba-92c6-bf49f325b5b4 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:27.032775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 970/1927] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:27.035538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9b0f0-3e4a-4153-92f5-6f012275c953 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:27.036052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9b0f0-3e4a-4153-92f5-6f012275c953 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:27.047766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9b0f0-3e4a-4153-92f5-6f012275c953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:27.094827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d3c9b0f0-3e4a-4153-92f5-6f012275c953 None None] MFA Rules not processed for user `f1e16c17cd0c4be690c693da833da8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:27.101372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d3c9b0f0-3e4a-4153-92f5-6f012275c953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:27.129843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07facc432e744ddbaafba4134ec73df'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:27.132557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07facc432e744ddbaafba4134ec73df'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:27.244704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 971/1928] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:27.379669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07facc432e744ddbaafba4134ec73df'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:27.400566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07facc432e744ddbaafba4134ec73df'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:27.402129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07facc432e744ddbaafba4134ec73df'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:27.429104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 958/1929] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:26 2022] DELETE /identity/v3/users/a07facc432e744ddbaafba4134ec73df => generated 0 bytes in 983 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:27.435339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba394-cfd0-46ec-8db6-ebde98d9c53c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:27.435771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba394-cfd0-46ec-8db6-ebde98d9c53c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:27.436118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba394-cfd0-46ec-8db6-ebde98d9c53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:27.458792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:27.460867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:27.462395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e63ba394-cfd0-46ec-8db6-ebde98d9c53c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:27.463714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e63ba394-cfd0-46ec-8db6-ebde98d9c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:27.547197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76627-06b9-470b-a15d-2254314d05d4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:27.568251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:27.568963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:27.569237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:27.569687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:27.653379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 972/1930] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:27.686186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ba394-cfd0-46ec-8db6-ebde98d9c53c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:27.689273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e63ba394-cfd0-46ec-8db6-ebde98d9c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:27.761691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:27.763981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:27.789976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 959/1931] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:27.965290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ba394-cfd0-46ec-8db6-ebde98d9c53c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:27.965290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:27.965290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:27.965290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:27.966048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:27.966048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:27.966048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:27.966048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 973/1932] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:28.770282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:28.899590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:28.906388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:28.906555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:28.906555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:28.931866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8e91f-42e8-400c-8323-8bdcc21d1d0b service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:28.941666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:28.942830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:28.943390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:28.943948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:28.947818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:28.967995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d9ebc-a816-4627-839b-a79be2cefdbe service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:28.978530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:28.980874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:28.982229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:28.983035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] PATH_INFO: `/v3/projects/8699f97b8d9843efbcf5521bfeae30b0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:28.987693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8699f97b8d9843efbcf5521bfeae30b0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:28.992742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:29.119850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:29.122024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:29.136497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 960/1933] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:29.247951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8699f97b8d9843efbcf5521bfeae30b0', 'request_id': 'req-3399474a-282b-49cb-a1ac-b43987bbdcc3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.260269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8699f97b8d9843efbcf5521bfeae30b0', 'request_id': 'req-3399474a-282b-49cb-a1ac-b43987bbdcc3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.275790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:29.352181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8699f97b8d9843efbcf5521bfeae30b0', 'request_id': 'req-3399474a-282b-49cb-a1ac-b43987bbdcc3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.369312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] The token cache is being invalidate because project 8699f97b8d9843efbcf5521bfeae30b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:29.369565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.370269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.371978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 974/1934] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:28 2022] DELETE /identity/v3/projects/8699f97b8d9843efbcf5521bfeae30b0 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:29.377186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:29.392739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217d976-ae39-48e6-b37c-d9fe7a14a94f service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:29.467091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:29.469113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:29.469602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:29.470087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] PATH_INFO: `/v3/users/32a70cdb408c4950a55696edf2921cfc` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:29.472411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a70cdb408c4950a55696edf2921cfc)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:29.478221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:29.766295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a70cdb408c4950a55696edf2921cfc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.767025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a70cdb408c4950a55696edf2921cfc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.888147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a70cdb408c4950a55696edf2921cfc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.899186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a70cdb408c4950a55696edf2921cfc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.901082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3279f860-2c1f-462f-b7b8-8f654a518b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a70cdb408c4950a55696edf2921cfc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:29.928952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 961/1935] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:29 2022] DELETE /identity/v3/users/32a70cdb408c4950a55696edf2921cfc => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:30.951660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:30.974306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:31.041466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399474a-282b-49cb-a1ac-b43987bbdcc3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:31.132826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:31.133609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:31.133898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:31.134178 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] PATH_INFO: `/v3/projects/3c397a12ca9c47a0ac4d78e016c93bbf` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:31.135656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c397a12ca9c47a0ac4d78e016c93bbf)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:31.138085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:31.405955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c397a12ca9c47a0ac4d78e016c93bbf', 'request_id': 'req-afdee7e1-a85d-42ed-a784-6059bf235f1f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:31.419668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c397a12ca9c47a0ac4d78e016c93bbf', 'request_id': 'req-afdee7e1-a85d-42ed-a784-6059bf235f1f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:31.606346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c397a12ca9c47a0ac4d78e016c93bbf', 'request_id': 'req-afdee7e1-a85d-42ed-a784-6059bf235f1f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:31.616815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] The token cache is being invalidate because project 3c397a12ca9c47a0ac4d78e016c93bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:31.617799 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:31.619551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-afdee7e1-a85d-42ed-a784-6059bf235f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:31.621971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 975/1936] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:30 2022] DELETE /identity/v3/projects/3c397a12ca9c47a0ac4d78e016c93bbf => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:31.631532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f56838-a9af-4dbe-a57e-c66b270e8f8c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:31.632090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f56838-a9af-4dbe-a57e-c66b270e8f8c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:31.633512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f56838-a9af-4dbe-a57e-c66b270e8f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:31.675707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-84f56838-a9af-4dbe-a57e-c66b270e8f8c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:31.677897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-84f56838-a9af-4dbe-a57e-c66b270e8f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:31.775961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 962/1937] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:31.786453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c220008-00e7-4d7d-af7e-dc59e08ee385 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:31.791426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c220008-00e7-4d7d-af7e-dc59e08ee385 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:31.791426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c220008-00e7-4d7d-af7e-dc59e08ee385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:31.828330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5c220008-00e7-4d7d-af7e-dc59e08ee385 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:31.830022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5c220008-00e7-4d7d-af7e-dc59e08ee385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:31.839518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 976/1938] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:31.846758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f56838-a9af-4dbe-a57e-c66b270e8f8c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:31.876024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f56838-a9af-4dbe-a57e-c66b270e8f8c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:31.876024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:31.876024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:31.876024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:31.876621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:31.876621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:31.876621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:31.876621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:31.897159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 963/1939] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:31.898309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c220008-00e7-4d7d-af7e-dc59e08ee385 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:31.913574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c220008-00e7-4d7d-af7e-dc59e08ee385 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:31.920192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa644-c977-4ba6-9342-16e24002d9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:31.928113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa644-c977-4ba6-9342-16e24002d9b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:31.928113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa644-c977-4ba6-9342-16e24002d9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:31.928113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa644-c977-4ba6-9342-16e24002d9b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:31.928113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa644-c977-4ba6-9342-16e24002d9b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:31.929931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa644-c977-4ba6-9342-16e24002d9b9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:31.998620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:32.000750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:32.027523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914679e-9d52-468b-8ff0-9c86c32822ba admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:32.039493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:32.039493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:32.039493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:32.039493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] PATH_INFO: `/v3/users/2c4dab23ea9a44bfb8db3d5d4abb8017` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:32.040916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c4dab23ea9a44bfb8db3d5d4abb8017)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:32.042944 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:32.069001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 977/1940] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:31 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:05:32.073837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa644-c977-4ba6-9342-16e24002d9b9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:32.231980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa644-c977-4ba6-9342-16e24002d9b9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:32.242658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:32.244333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:32.244755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:32.245530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:32.246780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:32.252076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:32.336360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c4dab23ea9a44bfb8db3d5d4abb8017'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:32.337770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c4dab23ea9a44bfb8db3d5d4abb8017'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:32.354634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:32.427415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 978/1941] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:32 2022] POST /identity/v3/users => generated 290 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:32.429462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:32.432641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:32.451747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c4dab23ea9a44bfb8db3d5d4abb8017'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:32.461330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c260f-140d-41ea-a617-cc024121aca2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:32.466798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b207-47fa-49dd-8209-2ae64fe7b73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:32.467532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b207-47fa-49dd-8209-2ae64fe7b73d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:32.467974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b207-47fa-49dd-8209-2ae64fe7b73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:32.468153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b207-47fa-49dd-8209-2ae64fe7b73d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:32.468997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c4dab23ea9a44bfb8db3d5d4abb8017'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:32.469194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63b207-47fa-49dd-8209-2ae64fe7b73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:32.470416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a57aca7a-8f27-4b09-b97d-1d7e00e32d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c4dab23ea9a44bfb8db3d5d4abb8017'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:32.471107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63b207-47fa-49dd-8209-2ae64fe7b73d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:32.486475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 979/1942] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:32.492803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b207-47fa-49dd-8209-2ae64fe7b73d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:32.507259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 964/1943] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:31 2022] DELETE /identity/v3/users/2c4dab23ea9a44bfb8db3d5d4abb8017 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:32.514753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b619-48be-4058-bb06-0b7e63885291 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:32.518026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b619-48be-4058-bb06-0b7e63885291 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:32.518641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b619-48be-4058-bb06-0b7e63885291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:32.524123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b207-47fa-49dd-8209-2ae64fe7b73d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:32.553107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:32.579494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2748b619-48be-4058-bb06-0b7e63885291 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:32.580778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2748b619-48be-4058-bb06-0b7e63885291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:32.653729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38061c-a0bf-4930-9df5-5e34987bf80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:32.655011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38061c-a0bf-4930-9df5-5e34987bf80e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:32.655919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38061c-a0bf-4930-9df5-5e34987bf80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:32.656595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38061c-a0bf-4930-9df5-5e34987bf80e admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:32.657611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 965/1944] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:32.658952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38061c-a0bf-4930-9df5-5e34987bf80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:32.662233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38061c-a0bf-4930-9df5-5e34987bf80e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:32.672252 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748b619-48be-4058-bb06-0b7e63885291 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:32.676540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2748b619-48be-4058-bb06-0b7e63885291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:32.685236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 980/1945] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:32.692669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38061c-a0bf-4930-9df5-5e34987bf80e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:32.734607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38061c-a0bf-4930-9df5-5e34987bf80e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:32.746935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1282a-9f11-4122-8e4e-d9a51b913edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:32.748795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1282a-9f11-4122-8e4e-d9a51b913edb admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:32.749345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1282a-9f11-4122-8e4e-d9a51b913edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:32.750483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1282a-9f11-4122-8e4e-d9a51b913edb admin admin] PATH_INFO: `/v3/projects/7197c5d17936406ba57f96fdd236733b/users/1a467b8bf56749a1a5018b9532d403bc/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:32.780255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1282a-9f11-4122-8e4e-d9a51b913edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7197c5d17936406ba57f96fdd236733b, user_id=1a467b8bf56749a1a5018b9532d403bc, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:32.788419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1282a-9f11-4122-8e4e-d9a51b913edb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:32.846861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748b619-48be-4058-bb06-0b7e63885291 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:32.852416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47783a4-aa69-4758-ba21-5334c79078ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:32.853251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47783a4-aa69-4758-ba21-5334c79078ff service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:32.853538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47783a4-aa69-4758-ba21-5334c79078ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:32.853824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47783a4-aa69-4758-ba21-5334c79078ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:32.867617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47783a4-aa69-4758-ba21-5334c79078ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:32.871204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47783a4-aa69-4758-ba21-5334c79078ff service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:32.887530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 966/1946] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:32.895710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 981/1947] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:32 2022] PUT /identity/v3/projects/7197c5d17936406ba57f96fdd236733b/users/1a467b8bf56749a1a5018b9532d403bc/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:32.908625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb040-1bcd-4339-8645-fe43024f4273 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:32.909085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb040-1bcd-4339-8645-fe43024f4273 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:32.910257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb040-1bcd-4339-8645-fe43024f4273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:32.960759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0eccb040-1bcd-4339-8645-fe43024f4273 None None] MFA Rules not processed for user `1a467b8bf56749a1a5018b9532d403bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:32.962896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0eccb040-1bcd-4339-8645-fe43024f4273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:33.164396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 967/1948] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:33.171472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1282a-9f11-4122-8e4e-d9a51b913edb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:33.298791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1282a-9f11-4122-8e4e-d9a51b913edb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:33.304883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26874430-1d3d-4879-ab13-73918aa4c355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:33.305649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874430-1d3d-4879-ab13-73918aa4c355 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:33.306052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874430-1d3d-4879-ab13-73918aa4c355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:33.306446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874430-1d3d-4879-ab13-73918aa4c355 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:33.307374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26874430-1d3d-4879-ab13-73918aa4c355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:33.309977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26874430-1d3d-4879-ab13-73918aa4c355 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:33.357810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:33.468306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 982/1949] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:33 2022] POST /identity/v3/projects => generated 377 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:33.470097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccb040-1bcd-4339-8645-fe43024f4273 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:33.483290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccb040-1bcd-4339-8645-fe43024f4273 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:33.620980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c549bc0-a244-4541-99b6-46596bb55ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:33.624299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c549bc0-a244-4541-99b6-46596bb55ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:33.624299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c549bc0-a244-4541-99b6-46596bb55ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:33.624299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c549bc0-a244-4541-99b6-46596bb55ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:33.626037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c549bc0-a244-4541-99b6-46596bb55ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:33.630983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c549bc0-a244-4541-99b6-46596bb55ee0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:33.778356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 968/1950] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:33 2022] POST /identity/v3/users => generated 289 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:05:33.794056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26874430-1d3d-4879-ab13-73918aa4c355 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:33.813027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26874430-1d3d-4879-ab13-73918aa4c355 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:33.820294 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae657c1-9975-496b-a0d6-e83781971c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:33.821162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae657c1-9975-496b-a0d6-e83781971c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:33.821585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae657c1-9975-496b-a0d6-e83781971c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:33.822032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae657c1-9975-496b-a0d6-e83781971c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:33.822981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae657c1-9975-496b-a0d6-e83781971c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:33.833347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae657c1-9975-496b-a0d6-e83781971c1f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:33.858174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 983/1951] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:33.863601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c549bc0-a244-4541-99b6-46596bb55ee0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:33.882279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c549bc0-a244-4541-99b6-46596bb55ee0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:33.889160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:33.889851 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:33.890616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:33.890905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:33.891975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:33.894434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:33.916940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 969/1952] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:33.918203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae657c1-9975-496b-a0d6-e83781971c1f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:33.933526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae657c1-9975-496b-a0d6-e83781971c1f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:33.939931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:33.941194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:33.941594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:33.941970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] PATH_INFO: `/v3/projects/bbe3934d8cb343ae92b5891fb5aecb32/users/6d329974fb794a948fcf3bee98b6d932/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:33.966367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe3934d8cb343ae92b5891fb5aecb32, user_id=6d329974fb794a948fcf3bee98b6d932, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:33.972171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:33.987411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:33.990230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:34.024663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d03d2-9296-4b46-b76d-f099a9d9bee3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:34.033893 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:34.035382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:34.035907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:34.036465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] PATH_INFO: `/v3/projects/432d6d42bc994329834ec0de75dade1b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:34.038249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=432d6d42bc994329834ec0de75dade1b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:34.041000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:34.101755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 984/1953] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:33 2022] PUT /identity/v3/projects/bbe3934d8cb343ae92b5891fb5aecb32/users/6d329974fb794a948fcf3bee98b6d932/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:34.103816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:34.109257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:34.219119 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432d6d42bc994329834ec0de75dade1b', 'request_id': 'req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.233625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432d6d42bc994329834ec0de75dade1b', 'request_id': 'req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.251359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432d6d42bc994329834ec0de75dade1b', 'request_id': 'req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.279961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468c5ac-8fac-48ce-9f07-5866e6e601eb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:34.391900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] The token cache is being invalidate because project 432d6d42bc994329834ec0de75dade1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:34.392201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.395938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7b3aba9b-92bf-4ef8-9eed-031a9950a0e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.398370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 970/1954] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:33 2022] DELETE /identity/v3/projects/432d6d42bc994329834ec0de75dade1b => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:34.399919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40584-0778-4f18-8d28-68fc3131636e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:34.402865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40584-0778-4f18-8d28-68fc3131636e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:34.403207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40584-0778-4f18-8d28-68fc3131636e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:34.416856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:34.417891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:34.418183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:34.418469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] PATH_INFO: `/v3/users/7cfaeed15e3f480bab4238b63f38ead4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:34.424099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cfaeed15e3f480bab4238b63f38ead4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:34.430797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:34.447770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-56c40584-0778-4f18-8d28-68fc3131636e None None] MFA Rules not processed for user `6d329974fb794a948fcf3bee98b6d932`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:34.449705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-56c40584-0778-4f18-8d28-68fc3131636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:34.568501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 971/1955] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:34.569292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c40584-0778-4f18-8d28-68fc3131636e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:34.570876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-56c40584-0778-4f18-8d28-68fc3131636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:34.590792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c40584-0778-4f18-8d28-68fc3131636e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:34.598884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:34.601948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:34.602826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:34.603934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] PATH_INFO: `/v3/users/7ae36b3a04814e5a97ba7ba743b780d9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:34.606323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae36b3a04814e5a97ba7ba743b780d9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:34.609656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:34.740738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cfaeed15e3f480bab4238b63f38ead4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.741636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cfaeed15e3f480bab4238b63f38ead4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.742866 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cfaeed15e3f480bab4238b63f38ead4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.795517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cfaeed15e3f480bab4238b63f38ead4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.825874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cfaeed15e3f480bab4238b63f38ead4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.837286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:34.924242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae36b3a04814e5a97ba7ba743b780d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.925813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae36b3a04814e5a97ba7ba743b780d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:34.953884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 985/1956] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:34 2022] DELETE /identity/v3/users/7cfaeed15e3f480bab4238b63f38ead4 => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:34.954899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:34.961868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:35.055463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceea91-bbf6-48d0-8f6b-1419c12d5bd0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:35.063030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:35.063030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:35.063575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:35.063575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] PATH_INFO: `/v3/users/01c53532edd34321ac16438229bfd3b0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:35.064978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c53532edd34321ac16438229bfd3b0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:35.067287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:35.089381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae36b3a04814e5a97ba7ba743b780d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:35.109613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae36b3a04814e5a97ba7ba743b780d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:35.110700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae36b3a04814e5a97ba7ba743b780d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:35.150797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 972/1957] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:34 2022] DELETE /identity/v3/users/7ae36b3a04814e5a97ba7ba743b780d9 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:35.150797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:35.158114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:35.329992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87489c7-c0ee-4dea-9104-8013e7a29b95 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:35.335647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fba4b4-5b68-42c2-a2ab-5bf89fbe2f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:35.336433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fba4b4-5b68-42c2-a2ab-5bf89fbe2f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:35.336788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fba4b4-5b68-42c2-a2ab-5bf89fbe2f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:35.337214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fba4b4-5b68-42c2-a2ab-5bf89fbe2f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:35.338046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fba4b4-5b68-42c2-a2ab-5bf89fbe2f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:35.340858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fba4b4-5b68-42c2-a2ab-5bf89fbe2f58 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:35.384651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c53532edd34321ac16438229bfd3b0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:35.385745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c53532edd34321ac16438229bfd3b0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:35.386471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c53532edd34321ac16438229bfd3b0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:35.401647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c53532edd34321ac16438229bfd3b0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:35.422090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c53532edd34321ac16438229bfd3b0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:35.547352 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 973/1958] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:35 2022] POST /identity/v3/projects => generated 377 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:35.552108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210185-ca11-46e0-b445-c9a856865cb2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:35.552500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210185-ca11-46e0-b445-c9a856865cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:35.552792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210185-ca11-46e0-b445-c9a856865cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:35.572582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b1210185-ca11-46e0-b445-c9a856865cb2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:35.573154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b1210185-ca11-46e0-b445-c9a856865cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:35.634940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 986/1959] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:34 2022] DELETE /identity/v3/users/01c53532edd34321ac16438229bfd3b0 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:35.635730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:35.639002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:35.688797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 974/1960] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:35.698438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd325ed-c2ec-441f-b2fb-d7e76b7dcc13 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:35.698438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd325ed-c2ec-441f-b2fb-d7e76b7dcc13 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:35.698438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd325ed-c2ec-441f-b2fb-d7e76b7dcc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:35.731997 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2fd325ed-c2ec-441f-b2fb-d7e76b7dcc13 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:35.738375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2fd325ed-c2ec-441f-b2fb-d7e76b7dcc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:35.772580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955f9ce-f5be-4249-be4e-c2f22cd46663 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:35.777654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44213ede-6347-458e-8eaf-0b21cca61acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:35.778464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44213ede-6347-458e-8eaf-0b21cca61acf admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:35.778854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44213ede-6347-458e-8eaf-0b21cca61acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:35.779257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44213ede-6347-458e-8eaf-0b21cca61acf admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:35.780126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44213ede-6347-458e-8eaf-0b21cca61acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:35.782219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44213ede-6347-458e-8eaf-0b21cca61acf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:35.816148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 975/1961] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:35.816148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd325ed-c2ec-441f-b2fb-d7e76b7dcc13 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:35.829398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2fd325ed-c2ec-441f-b2fb-d7e76b7dcc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:35.937775 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd325ed-c2ec-441f-b2fb-d7e76b7dcc13 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:35.941821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:35.942565 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:35.942799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:35.943073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:35.957061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 987/1962] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:35 2022] POST /identity/v3/users => generated 295 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:35.957984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44213ede-6347-458e-8eaf-0b21cca61acf admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:35.975665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:35.977276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44213ede-6347-458e-8eaf-0b21cca61acf admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:35.978328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:35.986302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7914800-49c9-4482-bdb4-76ceed4c6342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:35.987829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914800-49c9-4482-bdb4-76ceed4c6342 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:35.988227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914800-49c9-4482-bdb4-76ceed4c6342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:35.988684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914800-49c9-4482-bdb4-76ceed4c6342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.003564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 976/1963] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:36.003564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:36.006874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:36.011520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7914800-49c9-4482-bdb4-76ceed4c6342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:36.013919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7914800-49c9-4482-bdb4-76ceed4c6342 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:36.028390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 988/1964] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:36.029429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7914800-49c9-4482-bdb4-76ceed4c6342 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:36.030250 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd096b-aa88-49d8-8d41-b93929d4cf3a service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:36.035504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:36.036576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.036858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.037139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] PATH_INFO: `/v3/projects/74b568677fe44467b3b6bdefeb8c142a` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.042864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b568677fe44467b3b6bdefeb8c142a)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:36.043390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:36.046542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7914800-49c9-4482-bdb4-76ceed4c6342 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:36.052604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5291-0915-47eb-9c19-7b5876b19d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:36.053386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5291-0915-47eb-9c19-7b5876b19d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.058865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5291-0915-47eb-9c19-7b5876b19d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.058865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5291-0915-47eb-9c19-7b5876b19d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.058865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f5291-0915-47eb-9c19-7b5876b19d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:36.058865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f5291-0915-47eb-9c19-7b5876b19d3d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:36.071161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 989/1965] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:36.075710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5291-0915-47eb-9c19-7b5876b19d3d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:36.093941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5291-0915-47eb-9c19-7b5876b19d3d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:36.099309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e0dd0-5514-4d36-91aa-a3c5c333128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:36.100225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e0dd0-5514-4d36-91aa-a3c5c333128d admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.100509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e0dd0-5514-4d36-91aa-a3c5c333128d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.100786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e0dd0-5514-4d36-91aa-a3c5c333128d admin admin] PATH_INFO: `/v3/projects/b3f95c3881524b21b30554109fdc3734/users/628e721081044badaf54e3b89a77ff32/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.120582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e0dd0-5514-4d36-91aa-a3c5c333128d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f95c3881524b21b30554109fdc3734, user_id=628e721081044badaf54e3b89a77ff32, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:36.129519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e0dd0-5514-4d36-91aa-a3c5c333128d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:36.281922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 990/1966] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:36 2022] PUT /identity/v3/projects/b3f95c3881524b21b30554109fdc3734/users/628e721081044badaf54e3b89a77ff32/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:36.288458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e0dd0-5514-4d36-91aa-a3c5c333128d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:36.322871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b568677fe44467b3b6bdefeb8c142a', 'request_id': 'req-be5592d7-efa8-464f-b6d8-42015f9d76e2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:36.351136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b568677fe44467b3b6bdefeb8c142a', 'request_id': 'req-be5592d7-efa8-464f-b6d8-42015f9d76e2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:36.389069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b568677fe44467b3b6bdefeb8c142a', 'request_id': 'req-be5592d7-efa8-464f-b6d8-42015f9d76e2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:36.427074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:36.463585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e0dd0-5514-4d36-91aa-a3c5c333128d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:36.540222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] The token cache is being invalidate because project 74b568677fe44467b3b6bdefeb8c142a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:36.540552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:36.541217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-be5592d7-efa8-464f-b6d8-42015f9d76e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:36.543461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 977/1967] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:36 2022] DELETE /identity/v3/projects/74b568677fe44467b3b6bdefeb8c142a => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:36.549314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7959da-e274-4948-bfe7-93aa2adb49e6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.549756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7959da-e274-4948-bfe7-93aa2adb49e6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.550077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7959da-e274-4948-bfe7-93aa2adb49e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.575288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:36.575620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8c7959da-e274-4948-bfe7-93aa2adb49e6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:36.575891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.576201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.576480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.577256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:36.579342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8c7959da-e274-4948-bfe7-93aa2adb49e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:36.579625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:36.590461 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 978/1968] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:36.606672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 991/1969] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:36.608421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:36.615382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a934e69-ddbb-4743-b6cc-3ef74879eb34 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.615382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a934e69-ddbb-4743-b6cc-3ef74879eb34 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.615382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a934e69-ddbb-4743-b6cc-3ef74879eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.615684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:36.638921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7a934e69-ddbb-4743-b6cc-3ef74879eb34 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:36.642483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7a934e69-ddbb-4743-b6cc-3ef74879eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:36.654284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 979/1970] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:36.655040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111892-e72c-4520-b0be-640e7154396e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:36.682338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a934e69-ddbb-4743-b6cc-3ef74879eb34 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:36.683046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca733ef6-6edd-4d3b-9dd0-89e230bedea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:36.683046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca733ef6-6edd-4d3b-9dd0-89e230bedea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.683046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca733ef6-6edd-4d3b-9dd0-89e230bedea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.683046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca733ef6-6edd-4d3b-9dd0-89e230bedea2 admin admin] PATH_INFO: `/v3/projects/b3f95c3881524b21b30554109fdc3734/users/628e721081044badaf54e3b89a77ff32/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.685573 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca733ef6-6edd-4d3b-9dd0-89e230bedea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f95c3881524b21b30554109fdc3734, user_id=628e721081044badaf54e3b89a77ff32, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:36.686476 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a934e69-ddbb-4743-b6cc-3ef74879eb34 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:36.717543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:36.719040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.719504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.719939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.724169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:36.724555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca733ef6-6edd-4d3b-9dd0-89e230bedea2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:36.727775 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:36.731008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:36.763305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 992/1971] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:36 2022] PUT /identity/v3/projects/b3f95c3881524b21b30554109fdc3734/users/628e721081044badaf54e3b89a77ff32/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:36.771401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 980/1972] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:36.771529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a013b-60c8-42f6-9240-4eb9e50da7e0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.772341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a013b-60c8-42f6-9240-4eb9e50da7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.772341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a013b-60c8-42f6-9240-4eb9e50da7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.772512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:36.785456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684e8fc-39a6-41cc-8171-6774f1496270 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:36.813562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-364a013b-60c8-42f6-9240-4eb9e50da7e0 None None] MFA Rules not processed for user `628e721081044badaf54e3b89a77ff32`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:36.814829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-364a013b-60c8-42f6-9240-4eb9e50da7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:36.867030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe78e6-237c-45b5-9c3c-665931f030ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:36.867931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe78e6-237c-45b5-9c3c-665931f030ac admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:36.868272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe78e6-237c-45b5-9c3c-665931f030ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:36.868673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe78e6-237c-45b5-9c3c-665931f030ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:36.869679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe78e6-237c-45b5-9c3c-665931f030ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:36.872242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe78e6-237c-45b5-9c3c-665931f030ac admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:36.922445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 993/1973] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:36.938570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a013b-60c8-42f6-9240-4eb9e50da7e0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:36.940013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-364a013b-60c8-42f6-9240-4eb9e50da7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:37.012114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a013b-60c8-42f6-9240-4eb9e50da7e0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:37.021391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:37.022083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:37.022431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:37.022715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:37.028460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 981/1974] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:36 2022] POST /identity/v3/projects => generated 381 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:37.037122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:37.038692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:37.039422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe78e6-237c-45b5-9c3c-665931f030ac admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:37.068724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 994/1975] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:37.072153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:37.086197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:37.105813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409b532-eb05-456d-b7ab-4ca7543a026b service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:37.109768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:37.153915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe78e6-237c-45b5-9c3c-665931f030ac admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:37.158212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:37.159025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:37.159430 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:37.160069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:37.160621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:37.162706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:37.196108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:37.197019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:37.197305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:37.197574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] PATH_INFO: `/v3/projects/292f393c00b24047b4ac8620034ea64b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:37.198880 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] RBAC: Authorizing `identity:delete_project(project_id=292f393c00b24047b4ac8620034ea64b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:37.200862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:37.250020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 982/1976] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:37 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:37.250841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:37.256233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:37.275932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e7128-82be-4d3c-80d0-384d3681bf7f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:37.283028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:37.284639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:37.285152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:37.285651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] PATH_INFO: `/v3/projects/f063bf26a8ce42eebdc5faa690c55a49` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:37.287669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f063bf26a8ce42eebdc5faa690c55a49)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:37.291201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:37.355814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '292f393c00b24047b4ac8620034ea64b', 'request_id': 'req-1e653e21-cb9f-4db4-a5a6-4200da79820c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.366083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '292f393c00b24047b4ac8620034ea64b', 'request_id': 'req-1e653e21-cb9f-4db4-a5a6-4200da79820c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.424262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f063bf26a8ce42eebdc5faa690c55a49', 'request_id': 'req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.429963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f063bf26a8ce42eebdc5faa690c55a49', 'request_id': 'req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.439132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f063bf26a8ce42eebdc5faa690c55a49', 'request_id': 'req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.554609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '292f393c00b24047b4ac8620034ea64b', 'request_id': 'req-1e653e21-cb9f-4db4-a5a6-4200da79820c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.569355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] The token cache is being invalidate because project 292f393c00b24047b4ac8620034ea64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:37.569643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.570310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.572018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 995/1977] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:37 2022] DELETE /identity/v3/projects/292f393c00b24047b4ac8620034ea64b => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:37.572779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:37.574410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:37.639514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e653e21-cb9f-4db4-a5a6-4200da79820c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:37.702245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] The token cache is being invalidate because project f063bf26a8ce42eebdc5faa690c55a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:37.703357 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.704646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:37.706083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:37.707557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:37.707762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 983/1978] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:37 2022] DELETE /identity/v3/projects/f063bf26a8ce42eebdc5faa690c55a49 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:37.708020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:37.708708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:37.709662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:37.712192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:37.712713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:37.715268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:37.733185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 996/1979] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:37.735427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:37.739054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:37.757635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48073ea-d8a3-46de-b678-f8a701208f30 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:37.764124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:37.765679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:37.766023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:37.766353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] PATH_INFO: `/v3/users/ac343c4acdc74ceebdf876b4cf8deea8` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:37.767915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac343c4acdc74ceebdf876b4cf8deea8)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:37.771456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:37.824993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee462973-1c59-4120-bd8b-cbdf06cdb0b8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:37.832826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:37.834077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:37.834664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:37.835158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:37.838531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:37.951143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:37.954412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:37.972102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 984/1980] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:37.973613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:37.975877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:37.999669 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387a539-b556-4a6b-b8eb-b8b10da15c3d service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:38.008882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:38.010066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:38.010452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:38.010875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] PATH_INFO: `/v3/users/f1e16c17cd0c4be690c693da833da8bd` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:38.012456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e16c17cd0c4be690c693da833da8bd)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:38.015812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:38.077048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac343c4acdc74ceebdf876b4cf8deea8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.078766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac343c4acdc74ceebdf876b4cf8deea8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.079639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac343c4acdc74ceebdf876b4cf8deea8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.096955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac343c4acdc74ceebdf876b4cf8deea8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.106719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac343c4acdc74ceebdf876b4cf8deea8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.209970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 997/1981] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:37 2022] DELETE /identity/v3/users/ac343c4acdc74ceebdf876b4cf8deea8 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:38.213253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:38.213380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:38.258157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e16c17cd0c4be690c693da833da8bd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.259984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e16c17cd0c4be690c693da833da8bd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.325248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedd205-f0ed-4b79-990a-16929f7c28c0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:38.334858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:38.336205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:38.336621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:38.336948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:38.337979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:38.340079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:38.352504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 998/1982] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:38.357344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:38.359734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:38.362107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e16c17cd0c4be690c693da833da8bd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.372024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e16c17cd0c4be690c693da833da8bd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.372876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ef0d0ea2-c778-4b49-a52f-2c2867430782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e16c17cd0c4be690c693da833da8bd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:38.373900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de47874-a7d3-4486-be3a-0b2e8a5e0187 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:38.379068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8072361-67af-4473-9d0e-7d4c929fbcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:38.380097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8072361-67af-4473-9d0e-7d4c929fbcc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:38.380381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8072361-67af-4473-9d0e-7d4c929fbcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:38.380657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8072361-67af-4473-9d0e-7d4c929fbcc8 admin admin] PATH_INFO: `/v3/projects/e06ba22c68ac42e6b27270ff299462ff/users/c34fbaf3fd7c415e81aa1d501ca99e25/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:38.392340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 985/1983] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:37 2022] DELETE /identity/v3/users/f1e16c17cd0c4be690c693da833da8bd => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:38.397123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8072361-67af-4473-9d0e-7d4c929fbcc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06ba22c68ac42e6b27270ff299462ff, user_id=c34fbaf3fd7c415e81aa1d501ca99e25, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:38.399862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8072361-67af-4473-9d0e-7d4c929fbcc8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:38.405121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:38.523526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 999/1984] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:38 2022] PUT /identity/v3/projects/e06ba22c68ac42e6b27270ff299462ff/users/c34fbaf3fd7c415e81aa1d501ca99e25/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:38.535075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bab9c-5fce-489a-983e-514c02c7a3e4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:38.535423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bab9c-5fce-489a-983e-514c02c7a3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:38.535675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bab9c-5fce-489a-983e-514c02c7a3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:38.569533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fc3bab9c-5fce-489a-983e-514c02c7a3e4 None None] MFA Rules not processed for user `c34fbaf3fd7c415e81aa1d501ca99e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:38.570863 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fc3bab9c-5fce-489a-983e-514c02c7a3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:38.682430 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 986/1985] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:38.686143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8072361-67af-4473-9d0e-7d4c929fbcc8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:38.806533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8072361-67af-4473-9d0e-7d4c929fbcc8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:38.814257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:38.815447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:38.815934 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:38.816418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:38.817204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:38.820441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:38.879892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bab9c-5fce-489a-983e-514c02c7a3e4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:38.882390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fc3bab9c-5fce-489a-983e-514c02c7a3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:38.904198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bab9c-5fce-489a-983e-514c02c7a3e4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:38.913308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:38.914813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:38.915615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:38.916122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] PATH_INFO: `/v3/projects/f9749cd41118426a892faa6178245141` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:38.917884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9749cd41118426a892faa6178245141)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:38.924134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:39.016576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1000/1986] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:38 2022] POST /identity/v3/projects => generated 385 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:39.018920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:39.022238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:39.166600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9749cd41118426a892faa6178245141', 'request_id': 'req-631521a8-ea3d-4da3-9da6-467b315d9aa2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.184342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9749cd41118426a892faa6178245141', 'request_id': 'req-631521a8-ea3d-4da3-9da6-467b315d9aa2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.199601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c94d0-3711-4dcd-9895-fb00d2ab52f2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:39.204405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9749cd41118426a892faa6178245141', 'request_id': 'req-631521a8-ea3d-4da3-9da6-467b315d9aa2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.316867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] The token cache is being invalidate because project f9749cd41118426a892faa6178245141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:39.317952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.318541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.325833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 987/1987] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:38 2022] DELETE /identity/v3/projects/f9749cd41118426a892faa6178245141 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:39.325833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:39.326228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:39.326228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.326228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.326228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] PATH_INFO: `/v3/projects/40ee0ac154e14db186632adda367e833` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.327978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ee0ac154e14db186632adda367e833)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:39.331909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:39.334063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:39.354001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631521a8-ea3d-4da3-9da6-467b315d9aa2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:39.361467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636df1a1-2de1-41ce-a8b7-30e48277647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:39.362353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636df1a1-2de1-41ce-a8b7-30e48277647a admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.364645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636df1a1-2de1-41ce-a8b7-30e48277647a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.366824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636df1a1-2de1-41ce-a8b7-30e48277647a admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.367988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636df1a1-2de1-41ce-a8b7-30e48277647a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:39.370399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636df1a1-2de1-41ce-a8b7-30e48277647a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:39.523053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ee0ac154e14db186632adda367e833', 'request_id': 'req-319e7b87-05c7-498e-abdf-7a631f64f58a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.529618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ee0ac154e14db186632adda367e833', 'request_id': 'req-319e7b87-05c7-498e-abdf-7a631f64f58a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.540633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 988/1988] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:39 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:39.542177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d9586-99be-4549-b13a-ae39f900d4be None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.542473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d9586-99be-4549-b13a-ae39f900d4be None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.542750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d9586-99be-4549-b13a-ae39f900d4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.563788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e72d9586-99be-4549-b13a-ae39f900d4be None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:39.565037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e72d9586-99be-4549-b13a-ae39f900d4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:39.632315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 989/1989] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:39.633302 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d9586-99be-4549-b13a-ae39f900d4be None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:39.648121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d9586-99be-4549-b13a-ae39f900d4be None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:39.653514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3712639-1b79-4130-a9aa-aca2decbf668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:39.654194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3712639-1b79-4130-a9aa-aca2decbf668 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.654603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3712639-1b79-4130-a9aa-aca2decbf668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.655138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3712639-1b79-4130-a9aa-aca2decbf668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.665952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3712639-1b79-4130-a9aa-aca2decbf668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:39.666508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ee0ac154e14db186632adda367e833', 'request_id': 'req-319e7b87-05c7-498e-abdf-7a631f64f58a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.666603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3712639-1b79-4130-a9aa-aca2decbf668 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:39.669111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 990/1990] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:39.670598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9462b-91b1-4b61-a969-05b070550bcf None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.670919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9462b-91b1-4b61-a969-05b070550bcf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.671682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9462b-91b1-4b61-a969-05b070550bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.674837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] The token cache is being invalidate because project 40ee0ac154e14db186632adda367e833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:39.675128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.676184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:39.678144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1001/1991] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:39 2022] DELETE /identity/v3/projects/40ee0ac154e14db186632adda367e833 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:39.679244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:39.681068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:39.697045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e7b87-05c7-498e-abdf-7a631f64f58a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:39.702020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1cb9462b-91b1-4b61-a969-05b070550bcf None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:39.702227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b0a7d-8ea2-4ce6-9667-75288588042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:39.703410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b0a7d-8ea2-4ce6-9667-75288588042b admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.703594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1cb9462b-91b1-4b61-a969-05b070550bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:39.703716 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b0a7d-8ea2-4ce6-9667-75288588042b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.704032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b0a7d-8ea2-4ce6-9667-75288588042b admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.705204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b0a7d-8ea2-4ce6-9667-75288588042b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:39.707034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b0a7d-8ea2-4ce6-9667-75288588042b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:39.714783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 991/1992] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:39.718595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87c684-2698-4ee4-a76c-543ab3977e0f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.718595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87c684-2698-4ee4-a76c-543ab3977e0f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.718595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87c684-2698-4ee4-a76c-543ab3977e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.721872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1002/1993] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:39.727400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b0a7d-8ea2-4ce6-9667-75288588042b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:39.738888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b0a7d-8ea2-4ce6-9667-75288588042b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:39.741967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ea87c684-2698-4ee4-a76c-543ab3977e0f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:39.743200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ea87c684-2698-4ee4-a76c-543ab3977e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:39.753153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36782-332b-497a-8e78-36f3c3f8c84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:39.753153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36782-332b-497a-8e78-36f3c3f8c84a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.753153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36782-332b-497a-8e78-36f3c3f8c84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.753153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36782-332b-497a-8e78-36f3c3f8c84a admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.753824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36782-332b-497a-8e78-36f3c3f8c84a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:39.753824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36782-332b-497a-8e78-36f3c3f8c84a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:39.753824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36782-332b-497a-8e78-36f3c3f8c84a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:39.754248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 992/1994] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:39.754248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87c684-2698-4ee4-a76c-543ab3977e0f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:39.764536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87c684-2698-4ee4-a76c-543ab3977e0f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:39.769498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e92c98-5382-4bfe-a956-d1a0fe238f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:39.770482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e92c98-5382-4bfe-a956-d1a0fe238f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.770840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e92c98-5382-4bfe-a956-d1a0fe238f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.771191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e92c98-5382-4bfe-a956-d1a0fe238f67 admin admin] PATH_INFO: `/v3/projects/c1b13a1d309544a2acb650c89b4461dc/users/62c13b6c4c934fa5b2ed20bc3a521633/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.772702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1003/1995] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:39.777672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015813dd-48d9-4e48-a298-d1c196096820 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.777890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015813dd-48d9-4e48-a298-d1c196096820 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.778167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015813dd-48d9-4e48-a298-d1c196096820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.789966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e92c98-5382-4bfe-a956-d1a0fe238f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b13a1d309544a2acb650c89b4461dc, user_id=62c13b6c4c934fa5b2ed20bc3a521633, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:39.792732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e92c98-5382-4bfe-a956-d1a0fe238f67 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:39.798785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-015813dd-48d9-4e48-a298-d1c196096820 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:39.800071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-015813dd-48d9-4e48-a298-d1c196096820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:39.809470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1004/1996] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:39.810548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015813dd-48d9-4e48-a298-d1c196096820 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:39.821034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015813dd-48d9-4e48-a298-d1c196096820 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:39.825802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e889c-381e-43fe-b461-a53c0f50cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:39.826533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e889c-381e-43fe-b461-a53c0f50cc20 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.826842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e889c-381e-43fe-b461-a53c0f50cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.827170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e889c-381e-43fe-b461-a53c0f50cc20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.828027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e889c-381e-43fe-b461-a53c0f50cc20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:39.829932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e889c-381e-43fe-b461-a53c0f50cc20 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:39.917164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 993/1997] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:39 2022] PUT /identity/v3/projects/c1b13a1d309544a2acb650c89b4461dc/users/62c13b6c4c934fa5b2ed20bc3a521633/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:39.918118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e92c98-5382-4bfe-a956-d1a0fe238f67 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:39.951176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1005/1998] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:39 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:39.953269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c10a-a4ac-41b4-aeee-bf1b25ee4da7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:39.953269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c10a-a4ac-41b4-aeee-bf1b25ee4da7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:39.953269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c10a-a4ac-41b4-aeee-bf1b25ee4da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:39.984573 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7532c10a-a4ac-41b4-aeee-bf1b25ee4da7 None None] MFA Rules not processed for user `62c13b6c4c934fa5b2ed20bc3a521633`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:39.985849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7532c10a-a4ac-41b4-aeee-bf1b25ee4da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:39.990396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e92c98-5382-4bfe-a956-d1a0fe238f67 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.045863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:40.065119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1006/1999] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:40.065960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532c10a-a4ac-41b4-aeee-bf1b25ee4da7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.069496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.070190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.070470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.070741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.071548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:40.071904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.073545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.078430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532c10a-a4ac-41b4-aeee-bf1b25ee4da7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.084414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83463-de16-41a8-ba3c-b7f834e48976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.085107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83463-de16-41a8-ba3c-b7f834e48976 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.085441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83463-de16-41a8-ba3c-b7f834e48976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.085722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83463-de16-41a8-ba3c-b7f834e48976 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.086473 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f83463-de16-41a8-ba3c-b7f834e48976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.088533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f83463-de16-41a8-ba3c-b7f834e48976 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.090055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 994/2000] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:40.091013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.102051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb2cab-faf2-4c7e-a686-074def23fb69 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.107857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06716bd7-ab07-440f-9ce3-b079fd9ca006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.108540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06716bd7-ab07-440f-9ce3-b079fd9ca006 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.109342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06716bd7-ab07-440f-9ce3-b079fd9ca006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.109617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06716bd7-ab07-440f-9ce3-b079fd9ca006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.110314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06716bd7-ab07-440f-9ce3-b079fd9ca006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.112308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06716bd7-ab07-440f-9ce3-b079fd9ca006 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.229813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1007/2001] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:40 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:40.229813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83463-de16-41a8-ba3c-b7f834e48976 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.239172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 995/2002] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:40 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:40.239172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06716bd7-ab07-440f-9ce3-b079fd9ca006 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.241908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83463-de16-41a8-ba3c-b7f834e48976 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.319051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06716bd7-ab07-440f-9ce3-b079fd9ca006 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.321078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf781c5-0f4c-4224-8762-f946752574f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.321758 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf781c5-0f4c-4224-8762-f946752574f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.322046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf781c5-0f4c-4224-8762-f946752574f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.322324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf781c5-0f4c-4224-8762-f946752574f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.323024 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf781c5-0f4c-4224-8762-f946752574f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.324023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ca531-7429-4e52-be82-5093a065974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.324690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ca531-7429-4e52-be82-5093a065974f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.324981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ca531-7429-4e52-be82-5093a065974f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.325100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf781c5-0f4c-4224-8762-f946752574f9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.325256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ca531-7429-4e52-be82-5093a065974f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.325982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ca531-7429-4e52-be82-5093a065974f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.327696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ca531-7429-4e52-be82-5093a065974f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.342203 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 996/2003] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:40.342203 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ca531-7429-4e52-be82-5093a065974f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.350808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ca531-7429-4e52-be82-5093a065974f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.362725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf1d31-9572-4405-b23c-6074a23bddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.362725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf1d31-9572-4405-b23c-6074a23bddd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.362725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf1d31-9572-4405-b23c-6074a23bddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.362725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf1d31-9572-4405-b23c-6074a23bddd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.363087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf1d31-9572-4405-b23c-6074a23bddd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.363087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf1d31-9572-4405-b23c-6074a23bddd9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.454148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1008/2004] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:40 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:40.455469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf781c5-0f4c-4224-8762-f946752574f9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.492905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 997/2005] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:40 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:40.494354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf1d31-9572-4405-b23c-6074a23bddd9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.510211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf1d31-9572-4405-b23c-6074a23bddd9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.577089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf781c5-0f4c-4224-8762-f946752574f9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.582566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba00708-b458-4c3d-afda-87975e08859c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.583881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba00708-b458-4c3d-afda-87975e08859c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.584461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba00708-b458-4c3d-afda-87975e08859c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.584967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba00708-b458-4c3d-afda-87975e08859c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.586331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba00708-b458-4c3d-afda-87975e08859c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.589407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba00708-b458-4c3d-afda-87975e08859c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.589555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da371781-ccfa-41d4-8024-aa2356050638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.590682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da371781-ccfa-41d4-8024-aa2356050638 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.591071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da371781-ccfa-41d4-8024-aa2356050638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.591553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da371781-ccfa-41d4-8024-aa2356050638 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.592688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da371781-ccfa-41d4-8024-aa2356050638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.596709 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da371781-ccfa-41d4-8024-aa2356050638 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.614245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1009/2006] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:40.615831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba00708-b458-4c3d-afda-87975e08859c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.629459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba00708-b458-4c3d-afda-87975e08859c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.635849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e122e-5cd1-47da-9f84-3d43ac133bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.636727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e122e-5cd1-47da-9f84-3d43ac133bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.637033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e122e-5cd1-47da-9f84-3d43ac133bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.637456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e122e-5cd1-47da-9f84-3d43ac133bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.638206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e122e-5cd1-47da-9f84-3d43ac133bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.640012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e122e-5cd1-47da-9f84-3d43ac133bb7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.651323 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1010/2007] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:40.652056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e122e-5cd1-47da-9f84-3d43ac133bb7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.664075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e122e-5cd1-47da-9f84-3d43ac133bb7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.669491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b114d70-0e65-430b-8e24-2c79dceb71c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.670714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b114d70-0e65-430b-8e24-2c79dceb71c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.671248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b114d70-0e65-430b-8e24-2c79dceb71c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.671661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b114d70-0e65-430b-8e24-2c79dceb71c9 admin admin] PATH_INFO: `/v3/projects/d49d42de06d44a5596d9d4b0a98b96ab/users/c90097a563ee49c887eeef5f44df6471/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.690715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b114d70-0e65-430b-8e24-2c79dceb71c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49d42de06d44a5596d9d4b0a98b96ab, user_id=c90097a563ee49c887eeef5f44df6471, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.693575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b114d70-0e65-430b-8e24-2c79dceb71c9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.754534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 998/2008] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:40 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:40.754534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da371781-ccfa-41d4-8024-aa2356050638 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.774145 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da371781-ccfa-41d4-8024-aa2356050638 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.783765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1d93d-8c47-489c-8f56-1a299bafb301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.785073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1d93d-8c47-489c-8f56-1a299bafb301 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.785073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1d93d-8c47-489c-8f56-1a299bafb301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.785359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1d93d-8c47-489c-8f56-1a299bafb301 admin admin] PATH_INFO: `/v3/projects/1b07c800ff9243b5bc4a4f52ff0ffc28/users/5a2ccb41890146978ddd76f3077dbf7e/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.806348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1d93d-8c47-489c-8f56-1a299bafb301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b07c800ff9243b5bc4a4f52ff0ffc28, user_id=5a2ccb41890146978ddd76f3077dbf7e, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.812778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1d93d-8c47-489c-8f56-1a299bafb301 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.818537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1011/2009] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:40 2022] PUT /identity/v3/projects/d49d42de06d44a5596d9d4b0a98b96ab/users/c90097a563ee49c887eeef5f44df6471/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:40.819523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b114d70-0e65-430b-8e24-2c79dceb71c9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:40.890467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b114d70-0e65-430b-8e24-2c79dceb71c9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:40.899971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 999/2010] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:40 2022] PUT /identity/v3/projects/1b07c800ff9243b5bc4a4f52ff0ffc28/users/5a2ccb41890146978ddd76f3077dbf7e/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:40.903494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63301af-6734-4a3e-9d8a-d4d078b973cf None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.903494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63301af-6734-4a3e-9d8a-d4d078b973cf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.903494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63301af-6734-4a3e-9d8a-d4d078b973cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.942284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b63301af-6734-4a3e-9d8a-d4d078b973cf None None] MFA Rules not processed for user `c90097a563ee49c887eeef5f44df6471`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:40.944262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b63301af-6734-4a3e-9d8a-d4d078b973cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:40.973206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647872-9d4d-4855-805c-29d21c2a9df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:40.975275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647872-9d4d-4855-805c-29d21c2a9df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:40.975275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647872-9d4d-4855-805c-29d21c2a9df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:40.975275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647872-9d4d-4855-805c-29d21c2a9df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:40.975275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f647872-9d4d-4855-805c-29d21c2a9df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:40.976916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f647872-9d4d-4855-805c-29d21c2a9df3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:40.990200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1012/2011] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:40.991412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647872-9d4d-4855-805c-29d21c2a9df3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.001980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647872-9d4d-4855-805c-29d21c2a9df3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.007031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb779a-5547-4ac3-9bd9-925a865fb4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.007750 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb779a-5547-4ac3-9bd9-925a865fb4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.008072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb779a-5547-4ac3-9bd9-925a865fb4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.008312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb779a-5547-4ac3-9bd9-925a865fb4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.009068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb779a-5547-4ac3-9bd9-925a865fb4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.010722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb779a-5547-4ac3-9bd9-925a865fb4f7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.027026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1013/2012] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:41.028756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb779a-5547-4ac3-9bd9-925a865fb4f7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.043251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1000/2013] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:41.044131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63301af-6734-4a3e-9d8a-d4d078b973cf None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.047873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb779a-5547-4ac3-9bd9-925a865fb4f7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.055646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:41.056163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eda11-23db-4ad5-b726-4fe696db5f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.057334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eda11-23db-4ad5-b726-4fe696db5f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.057839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eda11-23db-4ad5-b726-4fe696db5f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.058326 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eda11-23db-4ad5-b726-4fe696db5f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.059748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eda11-23db-4ad5-b726-4fe696db5f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.063151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eda11-23db-4ad5-b726-4fe696db5f92 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.064784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63301af-6734-4a3e-9d8a-d4d078b973cf None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.072867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e6791-33c6-4e06-b946-b14bc89ccfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.074725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e6791-33c6-4e06-b946-b14bc89ccfca admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.075211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e6791-33c6-4e06-b946-b14bc89ccfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.075735 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e6791-33c6-4e06-b946-b14bc89ccfca admin admin] PATH_INFO: `/v3/projects/1b07c800ff9243b5bc4a4f52ff0ffc28/users/5a2ccb41890146978ddd76f3077dbf7e/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.080631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e6791-33c6-4e06-b946-b14bc89ccfca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b07c800ff9243b5bc4a4f52ff0ffc28, user_id=5a2ccb41890146978ddd76f3077dbf7e, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.085207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1014/2014] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:41.085207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eda11-23db-4ad5-b726-4fe696db5f92 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.086466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e6791-33c6-4e06-b946-b14bc89ccfca admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.098383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eda11-23db-4ad5-b726-4fe696db5f92 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.105470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1001/2015] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:41 2022] PUT /identity/v3/projects/1b07c800ff9243b5bc4a4f52ff0ffc28/users/5a2ccb41890146978ddd76f3077dbf7e/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:41.105470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e6791-33c6-4e06-b946-b14bc89ccfca admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.189817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e6791-33c6-4e06-b946-b14bc89ccfca admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.200581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d12ee-dfc3-49be-9b11-6bd39d4a0966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.200581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d12ee-dfc3-49be-9b11-6bd39d4a0966 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.200581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d12ee-dfc3-49be-9b11-6bd39d4a0966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.200880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11df76-4175-403d-95ed-65e2d1c7fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.200972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d12ee-dfc3-49be-9b11-6bd39d4a0966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.201049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11df76-4175-403d-95ed-65e2d1c7fabe admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.201118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d12ee-dfc3-49be-9b11-6bd39d4a0966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.201185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11df76-4175-403d-95ed-65e2d1c7fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.201185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11df76-4175-403d-95ed-65e2d1c7fabe admin admin] PATH_INFO: `/v3/projects/5046306e958c4eb5a6133a79b2c27df1/users/09aaaad871b04833aeeb51970bc62136/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.203249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d12ee-dfc3-49be-9b11-6bd39d4a0966 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.220462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11df76-4175-403d-95ed-65e2d1c7fabe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5046306e958c4eb5a6133a79b2c27df1, user_id=09aaaad871b04833aeeb51970bc62136, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.223150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11df76-4175-403d-95ed-65e2d1c7fabe admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.355401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1015/2016] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:41 2022] POST /identity/v3/projects => generated 389 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:41.355401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203045-0420-4d80-863e-db3bbadc4cc3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.355401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203045-0420-4d80-863e-db3bbadc4cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.355401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203045-0420-4d80-863e-db3bbadc4cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.360363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1002/2017] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:41 2022] PUT /identity/v3/projects/5046306e958c4eb5a6133a79b2c27df1/users/09aaaad871b04833aeeb51970bc62136/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:41.361424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11df76-4175-403d-95ed-65e2d1c7fabe admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.399528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8f203045-0420-4d80-863e-db3bbadc4cc3 None None] MFA Rules not processed for user `5a2ccb41890146978ddd76f3077dbf7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:41.401026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8f203045-0420-4d80-863e-db3bbadc4cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:41.473280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11df76-4175-403d-95ed-65e2d1c7fabe admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.494197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1016/2018] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:41.499925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74db77-ba50-4865-955c-ca8947d4edbc None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.500216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74db77-ba50-4865-955c-ca8947d4edbc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.500490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74db77-ba50-4865-955c-ca8947d4edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.531420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8f74db77-ba50-4865-955c-ca8947d4edbc None None] MFA Rules not processed for user `09aaaad871b04833aeeb51970bc62136`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:41.532698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8f74db77-ba50-4865-955c-ca8947d4edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:41.545906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaf34e-13b2-44c8-b27a-9035790a9d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.546721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaf34e-13b2-44c8-b27a-9035790a9d35 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.547011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaf34e-13b2-44c8-b27a-9035790a9d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.547325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaf34e-13b2-44c8-b27a-9035790a9d35 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.548037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beaf34e-13b2-44c8-b27a-9035790a9d35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.551626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beaf34e-13b2-44c8-b27a-9035790a9d35 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.611085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1017/2019] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:41.612015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74db77-ba50-4865-955c-ca8947d4edbc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.613591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8f74db77-ba50-4865-955c-ca8947d4edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:41.682396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1003/2020] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:41 2022] POST /identity/v3/users => generated 301 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:41.683641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaf34e-13b2-44c8-b27a-9035790a9d35 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.707493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaf34e-13b2-44c8-b27a-9035790a9d35 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.707607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74db77-ba50-4865-955c-ca8947d4edbc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.710271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b3be1-b51d-4424-8c44-7b801d6da2b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.711408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3be1-b51d-4424-8c44-7b801d6da2b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.711909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3be1-b51d-4424-8c44-7b801d6da2b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.712470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be655d-25a3-434c-b9a1-02a898c8cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.712708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3be1-b51d-4424-8c44-7b801d6da2b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.713494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be655d-25a3-434c-b9a1-02a898c8cc92 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.713921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be655d-25a3-434c-b9a1-02a898c8cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.714271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be655d-25a3-434c-b9a1-02a898c8cc92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.715000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be655d-25a3-434c-b9a1-02a898c8cc92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.717347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be655d-25a3-434c-b9a1-02a898c8cc92 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.723639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b3be1-b51d-4424-8c44-7b801d6da2b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.725205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b3be1-b51d-4424-8c44-7b801d6da2b7 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.737570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1018/2021] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:41.737979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b3be1-b51d-4424-8c44-7b801d6da2b7 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.749968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b3be1-b51d-4424-8c44-7b801d6da2b7 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.765257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c62e-16e8-4853-819d-d10bc06a9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.766093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c62e-16e8-4853-819d-d10bc06a9f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.766553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c62e-16e8-4853-819d-d10bc06a9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.766781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c62e-16e8-4853-819d-d10bc06a9f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.767672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678c62e-16e8-4853-819d-d10bc06a9f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.769755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678c62e-16e8-4853-819d-d10bc06a9f8b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.786170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1019/2022] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:41.786975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c62e-16e8-4853-819d-d10bc06a9f8b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.801065 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c62e-16e8-4853-819d-d10bc06a9f8b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.806942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fe3f-f1c5-4e0a-b552-18bbb6a2896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.807898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fe3f-f1c5-4e0a-b552-18bbb6a2896b admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.808184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fe3f-f1c5-4e0a-b552-18bbb6a2896b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.808462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fe3f-f1c5-4e0a-b552-18bbb6a2896b admin admin] PATH_INFO: `/v3/projects/78ae82c60ea941998c8f0c842985ba81/users/123e446fe9f24c5cbe6fd56c778466f9/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.833276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23fe3f-f1c5-4e0a-b552-18bbb6a2896b admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ae82c60ea941998c8f0c842985ba81, user_id=123e446fe9f24c5cbe6fd56c778466f9, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.836190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23fe3f-f1c5-4e0a-b552-18bbb6a2896b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:41.872294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1004/2023] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:41 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:41.874218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be655d-25a3-434c-b9a1-02a898c8cc92 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:41.981773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be655d-25a3-434c-b9a1-02a898c8cc92 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:41.987973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37af0f-8732-4d6e-9761-3013975afe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:41.989172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37af0f-8732-4d6e-9761-3013975afe67 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:41.989552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37af0f-8732-4d6e-9761-3013975afe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:41.989892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37af0f-8732-4d6e-9761-3013975afe67 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:41.990751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37af0f-8732-4d6e-9761-3013975afe67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:41.992895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37af0f-8732-4d6e-9761-3013975afe67 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.022442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1020/2024] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:41 2022] PUT /identity/v3/projects/78ae82c60ea941998c8f0c842985ba81/users/123e446fe9f24c5cbe6fd56c778466f9/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:42.026654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fe3f-f1c5-4e0a-b552-18bbb6a2896b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:42.125329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fe3f-f1c5-4e0a-b552-18bbb6a2896b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:42.133284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f1ce-40a2-47f2-baa0-0ba602e5ec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:42.134445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f1ce-40a2-47f2-baa0-0ba602e5ec34 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.135009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f1ce-40a2-47f2-baa0-0ba602e5ec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.135555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f1ce-40a2-47f2-baa0-0ba602e5ec34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.136944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14f1ce-40a2-47f2-baa0-0ba602e5ec34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:42.140182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14f1ce-40a2-47f2-baa0-0ba602e5ec34 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.156823 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1021/2025] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:42.161004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f1ce-40a2-47f2-baa0-0ba602e5ec34 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:42.171732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f1ce-40a2-47f2-baa0-0ba602e5ec34 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:42.177183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f2908-bb49-4913-ae55-fba0ce3970dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:42.178083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2908-bb49-4913-ae55-fba0ce3970dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.180772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2908-bb49-4913-ae55-fba0ce3970dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.180772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2908-bb49-4913-ae55-fba0ce3970dc admin admin] PATH_INFO: `/v3/projects/78ae82c60ea941998c8f0c842985ba81/users/123e446fe9f24c5cbe6fd56c778466f9/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.181043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1005/2026] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:41 2022] POST /identity/v3/users => generated 299 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:42.182166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f2908-bb49-4913-ae55-fba0ce3970dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ae82c60ea941998c8f0c842985ba81, user_id=123e446fe9f24c5cbe6fd56c778466f9, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:42.183716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37af0f-8732-4d6e-9761-3013975afe67 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:42.185122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f2908-bb49-4913-ae55-fba0ce3970dc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.195111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37af0f-8732-4d6e-9761-3013975afe67 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:42.197145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1022/2027] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:42 2022] PUT /identity/v3/projects/78ae82c60ea941998c8f0c842985ba81/users/123e446fe9f24c5cbe6fd56c778466f9/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:42.208131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212173a2-5e3b-4f5c-820f-16f52d1ad78e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.208536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212173a2-5e3b-4f5c-820f-16f52d1ad78e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.208725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212173a2-5e3b-4f5c-820f-16f52d1ad78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.237246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-212173a2-5e3b-4f5c-820f-16f52d1ad78e None None] MFA Rules not processed for user `123e446fe9f24c5cbe6fd56c778466f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:42.238419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-212173a2-5e3b-4f5c-820f-16f52d1ad78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:42.257132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feda26-6e42-4ec4-9bb3-42fe908d05e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:42.257924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feda26-6e42-4ec4-9bb3-42fe908d05e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.258212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feda26-6e42-4ec4-9bb3-42fe908d05e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.258488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feda26-6e42-4ec4-9bb3-42fe908d05e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.259241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feda26-6e42-4ec4-9bb3-42fe908d05e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:42.261024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feda26-6e42-4ec4-9bb3-42fe908d05e7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.273439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1006/2028] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:42.280223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feda26-6e42-4ec4-9bb3-42fe908d05e7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:42.291840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feda26-6e42-4ec4-9bb3-42fe908d05e7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:42.299189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c56f8-b4da-42c4-b601-8b980b45128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:42.299713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c56f8-b4da-42c4-b601-8b980b45128f admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.300020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c56f8-b4da-42c4-b601-8b980b45128f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.300277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c56f8-b4da-42c4-b601-8b980b45128f admin admin] PATH_INFO: `/v3/projects/83615912b4384529aa09e0093c45e660/users/fa40339589d04a24af4c7be0bf7f1139/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.321745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c56f8-b4da-42c4-b601-8b980b45128f admin admin] RBAC: Authorizing `identity:create_grant(project_id=83615912b4384529aa09e0093c45e660, user_id=fa40339589d04a24af4c7be0bf7f1139, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:42.324336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1023/2029] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:42.325042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c56f8-b4da-42c4-b601-8b980b45128f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.337402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212173a2-5e3b-4f5c-820f-16f52d1ad78e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:42.339045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-212173a2-5e3b-4f5c-820f-16f52d1ad78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:42.409237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:42.413102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212173a2-5e3b-4f5c-820f-16f52d1ad78e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:42.416923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8124641-70b1-4e9c-b90d-99eb1c5c3fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:42.417598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8124641-70b1-4e9c-b90d-99eb1c5c3fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.417877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8124641-70b1-4e9c-b90d-99eb1c5c3fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.418166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8124641-70b1-4e9c-b90d-99eb1c5c3fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.427680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8124641-70b1-4e9c-b90d-99eb1c5c3fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:42.429099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8124641-70b1-4e9c-b90d-99eb1c5c3fe3 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.439184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1024/2030] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:42.446635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1007/2031] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:42 2022] PUT /identity/v3/projects/83615912b4384529aa09e0093c45e660/users/fa40339589d04a24af4c7be0bf7f1139/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:42.451023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8124641-70b1-4e9c-b90d-99eb1c5c3fe3 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:42.468874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8124641-70b1-4e9c-b90d-99eb1c5c3fe3 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:42.546305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae9970-ab60-4d3d-a65b-ae50d7a6cecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:42.547049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9970-ab60-4d3d-a65b-ae50d7a6cecb admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.547355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9970-ab60-4d3d-a65b-ae50d7a6cecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.547785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9970-ab60-4d3d-a65b-ae50d7a6cecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.548619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae9970-ab60-4d3d-a65b-ae50d7a6cecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:42.554386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae9970-ab60-4d3d-a65b-ae50d7a6cecb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.565385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1025/2032] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:42.567536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c56f8-b4da-42c4-b601-8b980b45128f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:42.603456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c56f8-b4da-42c4-b601-8b980b45128f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:42.619811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75faeb4-4aab-49f7-9466-1b1626e746ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:42.620773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75faeb4-4aab-49f7-9466-1b1626e746ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.621073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75faeb4-4aab-49f7-9466-1b1626e746ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.621619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75faeb4-4aab-49f7-9466-1b1626e746ef admin admin] PATH_INFO: `/v3/projects/83615912b4384529aa09e0093c45e660/users/fa40339589d04a24af4c7be0bf7f1139/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.628273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75faeb4-4aab-49f7-9466-1b1626e746ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=83615912b4384529aa09e0093c45e660, user_id=fa40339589d04a24af4c7be0bf7f1139, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:42.632037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75faeb4-4aab-49f7-9466-1b1626e746ef admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.649070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1008/2033] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:42 2022] PUT /identity/v3/projects/83615912b4384529aa09e0093c45e660/users/fa40339589d04a24af4c7be0bf7f1139/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:42.666004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32233957-012b-40e0-a276-e97ceae66897 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.666154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32233957-012b-40e0-a276-e97ceae66897 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.666658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32233957-012b-40e0-a276-e97ceae66897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.697395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75faeb4-4aab-49f7-9466-1b1626e746ef admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:42.716598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-32233957-012b-40e0-a276-e97ceae66897 None None] MFA Rules not processed for user `fa40339589d04a24af4c7be0bf7f1139`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:42.717918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-32233957-012b-40e0-a276-e97ceae66897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:42.787329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75faeb4-4aab-49f7-9466-1b1626e746ef admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:42.802667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:42.803614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:42.803927 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:42.804213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] PATH_INFO: `/v3/users/c90097a563ee49c887eeef5f44df6471` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:42.805560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90097a563ee49c887eeef5f44df6471)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:42.807706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:42.833069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1026/2034] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:42.995192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32233957-012b-40e0-a276-e97ceae66897 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:43.061651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32233957-012b-40e0-a276-e97ceae66897 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:43.065943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d04e2d-1c48-4268-82cb-816610c00706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:43.066642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04e2d-1c48-4268-82cb-816610c00706 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:43.066933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04e2d-1c48-4268-82cb-816610c00706 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:43.067192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04e2d-1c48-4268-82cb-816610c00706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:43.079937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d04e2d-1c48-4268-82cb-816610c00706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:43.081793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d04e2d-1c48-4268-82cb-816610c00706 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:43.092937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1027/2035] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:43.109421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90097a563ee49c887eeef5f44df6471'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.110114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90097a563ee49c887eeef5f44df6471'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.180665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90097a563ee49c887eeef5f44df6471'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.189208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90097a563ee49c887eeef5f44df6471'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.190003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90097a563ee49c887eeef5f44df6471'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.220091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1009/2036] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:42 2022] DELETE /identity/v3/users/c90097a563ee49c887eeef5f44df6471 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:43.227516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca31c7f-d289-4444-803c-53b36727fb8f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:43.227832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca31c7f-d289-4444-803c-53b36727fb8f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:43.228111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca31c7f-d289-4444-803c-53b36727fb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:43.240964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:43.245566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:43.257864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-aca31c7f-d289-4444-803c-53b36727fb8f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:43.259417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-aca31c7f-d289-4444-803c-53b36727fb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:43.364274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49c5e2-a514-416f-94d1-57160a21bc22 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:43.367382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1028/2037] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:43.380185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca31c7f-d289-4444-803c-53b36727fb8f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:43.381762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-aca31c7f-d289-4444-803c-53b36727fb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:43.386217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:43.387120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:43.387507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:43.387770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] PATH_INFO: `/v3/users/1a467b8bf56749a1a5018b9532d403bc` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:43.389239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a467b8bf56749a1a5018b9532d403bc)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:43.391799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:43.465477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca31c7f-d289-4444-803c-53b36727fb8f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:43.470501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:43.471455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:43.471776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:43.472113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:43.482801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:43.484311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:43.495470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1029/2038] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:43.496398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:43.497793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:43.578617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57702910-2f79-4572-88f1-5835dea14607 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:43.582922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c9d0-d9a4-40c2-b065-02ed2aaa2dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:43.588266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c9d0-d9a4-40c2-b065-02ed2aaa2dbe service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:43.588266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c9d0-d9a4-40c2-b065-02ed2aaa2dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:43.588266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c9d0-d9a4-40c2-b065-02ed2aaa2dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:43.588266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b7b7c9d0-d9a4-40c2-b065-02ed2aaa2dbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:43.661150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7c9d0-d9a4-40c2-b065-02ed2aaa2dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:43.662896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7c9d0-d9a4-40c2-b065-02ed2aaa2dbe service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:43.673385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1030/2039] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:43.703068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a467b8bf56749a1a5018b9532d403bc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.703884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a467b8bf56749a1a5018b9532d403bc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.737675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:43.854289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a467b8bf56749a1a5018b9532d403bc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.865058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a467b8bf56749a1a5018b9532d403bc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.865978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a467b8bf56749a1a5018b9532d403bc'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:43.895891 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1010/2040] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:43 2022] DELETE /identity/v3/users/1a467b8bf56749a1a5018b9532d403bc => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:43.901294 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae20670-624d-4221-9fb5-60e450f814aa None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:43.901743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae20670-624d-4221-9fb5-60e450f814aa None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:43.902224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae20670-624d-4221-9fb5-60e450f814aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:43.925480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3ae20670-624d-4221-9fb5-60e450f814aa None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:43.927672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3ae20670-624d-4221-9fb5-60e450f814aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:43.996955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1031/2041] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:44.007536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:44.009073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:44.076765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d20dd-457e-4261-be2c-f905c2944d43 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:44.083298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacd105-2694-43b3-b572-fde25af6c9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:44.084331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacd105-2694-43b3-b572-fde25af6c9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:44.084838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacd105-2694-43b3-b572-fde25af6c9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:44.085465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacd105-2694-43b3-b572-fde25af6c9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:44.102373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacd105-2694-43b3-b572-fde25af6c9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:44.104525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacd105-2694-43b3-b572-fde25af6c9a4 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:44.120131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1011/2042] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:44.208198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae20670-624d-4221-9fb5-60e450f814aa None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:44.211161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3ae20670-624d-4221-9fb5-60e450f814aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:44.229631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae20670-624d-4221-9fb5-60e450f814aa None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:44.242288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:44.244489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:44.244991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:44.245784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] PATH_INFO: `/v3/projects/d49d42de06d44a5596d9d4b0a98b96ab` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:44.247924 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49d42de06d44a5596d9d4b0a98b96ab)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:44.251590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:44.449850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49d42de06d44a5596d9d4b0a98b96ab', 'request_id': 'req-f8a858eb-6be7-4f87-9109-249c75a722f2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:44.461045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49d42de06d44a5596d9d4b0a98b96ab', 'request_id': 'req-f8a858eb-6be7-4f87-9109-249c75a722f2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:44.540450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49d42de06d44a5596d9d4b0a98b96ab', 'request_id': 'req-f8a858eb-6be7-4f87-9109-249c75a722f2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:44.572158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] The token cache is being invalidate because project d49d42de06d44a5596d9d4b0a98b96ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:44.572761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:44.573962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:44.581228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1032/2043] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:44 2022] DELETE /identity/v3/projects/d49d42de06d44a5596d9d4b0a98b96ab => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:44.584666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacd105-2694-43b3-b572-fde25af6c9a4 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:44.596391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacd105-2694-43b3-b572-fde25af6c9a4 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:44.667117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:44.668714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:44.669011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:44.669285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] PATH_INFO: `/v3/users/123e446fe9f24c5cbe6fd56c778466f9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:44.670793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=123e446fe9f24c5cbe6fd56c778466f9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:44.673196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:44.863516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:44.865039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:44.879025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a858eb-6be7-4f87-9109-249c75a722f2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:44.884540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:44.893534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:44.893534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:44.893534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] PATH_INFO: `/v3/projects/7197c5d17936406ba57f96fdd236733b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:44.893534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7197c5d17936406ba57f96fdd236733b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:44.893534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:44.973168 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123e446fe9f24c5cbe6fd56c778466f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:44.973968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123e446fe9f24c5cbe6fd56c778466f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.087172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7197c5d17936406ba57f96fdd236733b', 'request_id': 'req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.091984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123e446fe9f24c5cbe6fd56c778466f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.097693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7197c5d17936406ba57f96fdd236733b', 'request_id': 'req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.103507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123e446fe9f24c5cbe6fd56c778466f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.104258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123e446fe9f24c5cbe6fd56c778466f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.124685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1012/2044] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:44 2022] DELETE /identity/v3/users/123e446fe9f24c5cbe6fd56c778466f9 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:45.125535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:45.126889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:45.209867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c968fe-dfce-42c8-a1cf-b559f9690f4b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:45.213617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7197c5d17936406ba57f96fdd236733b', 'request_id': 'req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.216144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:45.216702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:45.217040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:45.217421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:45.217661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] PATH_INFO: `/v3/users/09aaaad871b04833aeeb51970bc62136` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:45.220021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=09aaaad871b04833aeeb51970bc62136)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:45.222607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:45.230386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] The token cache is being invalidate because project 7197c5d17936406ba57f96fdd236733b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:45.230919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.231880 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.239245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1033/2045] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:44 2022] DELETE /identity/v3/projects/7197c5d17936406ba57f96fdd236733b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:45.240431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:45.243977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:45.357911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78face-4864-4b3f-a9f0-0c356c8d3fd2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:45.363922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:45.365024 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:45.365317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:45.365686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] PATH_INFO: `/v3/users/6d329974fb794a948fcf3bee98b6d932` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:45.367128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d329974fb794a948fcf3bee98b6d932)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:45.370609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:45.497889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09aaaad871b04833aeeb51970bc62136'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.498587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09aaaad871b04833aeeb51970bc62136'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.596259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d329974fb794a948fcf3bee98b6d932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.597272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d329974fb794a948fcf3bee98b6d932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.597858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d329974fb794a948fcf3bee98b6d932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.609530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09aaaad871b04833aeeb51970bc62136'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.614963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d329974fb794a948fcf3bee98b6d932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.624654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d329974fb794a948fcf3bee98b6d932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.632978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09aaaad871b04833aeeb51970bc62136'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.634279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab02d31e-9f23-4ee2-baba-67f9aab062ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09aaaad871b04833aeeb51970bc62136'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:45.662193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1013/2046] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:45 2022] DELETE /identity/v3/users/09aaaad871b04833aeeb51970bc62136 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:45.668117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb030494-26fb-484b-bf0e-8276fbf7abb6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:45.668976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb030494-26fb-484b-bf0e-8276fbf7abb6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:45.669635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb030494-26fb-484b-bf0e-8276fbf7abb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:45.696916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-cb030494-26fb-484b-bf0e-8276fbf7abb6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:45.696916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cb030494-26fb-484b-bf0e-8276fbf7abb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:45.773163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1014/2047] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:45.794985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb030494-26fb-484b-bf0e-8276fbf7abb6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:45.801433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cb030494-26fb-484b-bf0e-8276fbf7abb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:45.804581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1034/2048] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:45 2022] DELETE /identity/v3/users/6d329974fb794a948fcf3bee98b6d932 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:45.880017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb030494-26fb-484b-bf0e-8276fbf7abb6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:45.946937 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:45.947696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:45.947986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:45.948324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:46.018440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:46.020299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:46.036405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1015/2049] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:46.435154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:46.437830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:46.460676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:46.463447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a502ca-b549-431b-8adc-dd425a0921f7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:46.463746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:46.477422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:46.478432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:46.478722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:46.479003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] PATH_INFO: `/v3/users/430ca4a6af054228802290aa876120eb` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:46.480615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=430ca4a6af054228802290aa876120eb)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:46.482852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:46.487995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:46.506499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2e5f-ccfe-4319-b903-3945eb3bf7b8 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:46.524710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:46.525684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:46.525975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:46.526264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] PATH_INFO: `/v3/projects/78ae82c60ea941998c8f0c842985ba81` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:46.527574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ae82c60ea941998c8f0c842985ba81)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:46.529770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:46.762100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '430ca4a6af054228802290aa876120eb'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.762826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '430ca4a6af054228802290aa876120eb'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.763569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '430ca4a6af054228802290aa876120eb'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.768757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ae82c60ea941998c8f0c842985ba81', 'request_id': 'req-7769d6ae-2242-4b1d-ba45-83388185bb78'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.775931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ae82c60ea941998c8f0c842985ba81', 'request_id': 'req-7769d6ae-2242-4b1d-ba45-83388185bb78'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.783189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '430ca4a6af054228802290aa876120eb'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.783665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ae82c60ea941998c8f0c842985ba81', 'request_id': 'req-7769d6ae-2242-4b1d-ba45-83388185bb78'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.791527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '430ca4a6af054228802290aa876120eb'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.968540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] The token cache is being invalidate because project 78ae82c60ea941998c8f0c842985ba81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:46.968540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.968540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:46.968540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1016/2050] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:46 2022] DELETE /identity/v3/projects/78ae82c60ea941998c8f0c842985ba81 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:46.968540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:46.969285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:47.055497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769d6ae-2242-4b1d-ba45-83388185bb78 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:47.061807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:47.063363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:47.063686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:47.064009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] PATH_INFO: `/v3/projects/bbe3934d8cb343ae92b5891fb5aecb32` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:47.065358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe3934d8cb343ae92b5891fb5aecb32)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:47.067479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:47.079448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1035/2051] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:46 2022] DELETE /identity/v3/users/430ca4a6af054228802290aa876120eb => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:47.081362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:47.084237 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:47.168310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef499d2-7202-43e8-8f12-cffa17d444b1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:47.173936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:47.174897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:47.175183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:47.175503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] PATH_INFO: `/v3/projects/5046306e958c4eb5a6133a79b2c27df1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:47.176779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5046306e958c4eb5a6133a79b2c27df1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:47.179089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:47.214809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe3934d8cb343ae92b5891fb5aecb32', 'request_id': 'req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.222750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe3934d8cb343ae92b5891fb5aecb32', 'request_id': 'req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.230189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe3934d8cb343ae92b5891fb5aecb32', 'request_id': 'req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.291112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5046306e958c4eb5a6133a79b2c27df1', 'request_id': 'req-90da205f-3917-4549-889e-c6effcfa87b6'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.297819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5046306e958c4eb5a6133a79b2c27df1', 'request_id': 'req-90da205f-3917-4549-889e-c6effcfa87b6'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.328651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] The token cache is being invalidate because project bbe3934d8cb343ae92b5891fb5aecb32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:47.328992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.329688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4a69871-c7a6-4cf1-aca1-e5e62bb96ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.331448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1017/2052] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:46 2022] DELETE /identity/v3/projects/bbe3934d8cb343ae92b5891fb5aecb32 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:47.332849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc4f94-a441-4ce1-8bfb-8dc663f96ed9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:47.333195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc4f94-a441-4ce1-8bfb-8dc663f96ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:47.333542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc4f94-a441-4ce1-8bfb-8dc663f96ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:47.359828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f6cc4f94-a441-4ce1-8bfb-8dc663f96ed9 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:47.362412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f6cc4f94-a441-4ce1-8bfb-8dc663f96ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:47.411785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5046306e958c4eb5a6133a79b2c27df1', 'request_id': 'req-90da205f-3917-4549-889e-c6effcfa87b6'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.436560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] The token cache is being invalidate because project 5046306e958c4eb5a6133a79b2c27df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:47.438184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.438599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-90da205f-3917-4549-889e-c6effcfa87b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.442583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1036/2053] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:47 2022] DELETE /identity/v3/projects/5046306e958c4eb5a6133a79b2c27df1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:47.445427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097ea0-c9bf-4eb6-aaf2-0c0b3123d735 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:47.446406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097ea0-c9bf-4eb6-aaf2-0c0b3123d735 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:47.447276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097ea0-c9bf-4eb6-aaf2-0c0b3123d735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:47.482079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1018/2054] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:47.484841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0c097ea0-c9bf-4eb6-aaf2-0c0b3123d735 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:47.485083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc4f94-a441-4ce1-8bfb-8dc663f96ed9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:47.487353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0c097ea0-c9bf-4eb6-aaf2-0c0b3123d735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:47.488032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f6cc4f94-a441-4ce1-8bfb-8dc663f96ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:47.501159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1037/2055] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:47.503195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c097ea0-c9bf-4eb6-aaf2-0c0b3123d735 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:47.504470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc4f94-a441-4ce1-8bfb-8dc663f96ed9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:47.505490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0c097ea0-c9bf-4eb6-aaf2-0c0b3123d735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:47.511369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:47.512501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:47.512931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:47.513544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] PATH_INFO: `/v3/users/628e721081044badaf54e3b89a77ff32` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:47.515132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=628e721081044badaf54e3b89a77ff32)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:47.517371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:47.522987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c097ea0-c9bf-4eb6-aaf2-0c0b3123d735 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:47.534560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:47.536464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:47.537697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:47.538261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] PATH_INFO: `/v3/users/fa40339589d04a24af4c7be0bf7f1139` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:47.540304 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa40339589d04a24af4c7be0bf7f1139)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:47.543366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:47.761525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628e721081044badaf54e3b89a77ff32'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.762271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628e721081044badaf54e3b89a77ff32'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.870627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628e721081044badaf54e3b89a77ff32'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.879766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628e721081044badaf54e3b89a77ff32'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.880649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-dd77f964-2ce5-40bd-ab6d-3677cfc9b683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628e721081044badaf54e3b89a77ff32'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.891719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa40339589d04a24af4c7be0bf7f1139'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.892475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa40339589d04a24af4c7be0bf7f1139'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.893096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa40339589d04a24af4c7be0bf7f1139'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.902505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1019/2056] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:47 2022] DELETE /identity/v3/users/628e721081044badaf54e3b89a77ff32 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:47.903261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:47.903822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6fa9-da81-4961-acf8-e51f74f56f39 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:47.904112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6fa9-da81-4961-acf8-e51f74f56f39 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:47.904557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6fa9-da81-4961-acf8-e51f74f56f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:47.918426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa40339589d04a24af4c7be0bf7f1139'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.928575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa40339589d04a24af4c7be0bf7f1139'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:47.933270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ee5c6fa9-da81-4961-acf8-e51f74f56f39 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:47.934462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ee5c6fa9-da81-4961-acf8-e51f74f56f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:48.007525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1020/2057] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:48.008733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c6fa9-da81-4961-acf8-e51f74f56f39 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:48.014355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ee5c6fa9-da81-4961-acf8-e51f74f56f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:48.097816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1038/2058] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:47 2022] DELETE /identity/v3/users/fa40339589d04a24af4c7be0bf7f1139 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:48.097816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:48.182504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c6fa9-da81-4961-acf8-e51f74f56f39 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:48.247998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3ebc3-8bd9-4769-bb28-e8e3d4d3df66 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:48.263051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:48.272202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:48.272537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:48.272776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:48.273617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:48.274096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:48.275850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:48.291045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:48.292351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:48.292703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:48.292988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:48.296627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:48.302659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1039/2059] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:48.308489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:48.332182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d563e2-1db8-4aa8-9b6a-be3431e7561f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:48.341100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:48.343242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:48.348004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:48.348685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:48.349143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:48.349439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:48.350402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:48.352178 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:48.363116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1021/2060] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:48.527723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1040/2061] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:48 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:48.533189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:48.545114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f2525-8516-4ac9-ac0b-c633313d69f5 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:48.631385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:48.636394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:48.642082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794e5e8-925f-446d-bba2-1d0634018e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:48.642783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794e5e8-925f-446d-bba2-1d0634018e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:48.643070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794e5e8-925f-446d-bba2-1d0634018e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:48.643383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794e5e8-925f-446d-bba2-1d0634018e1e admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:48.644103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794e5e8-925f-446d-bba2-1d0634018e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:48.647548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794e5e8-925f-446d-bba2-1d0634018e1e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:48.663847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c47c-ccf2-4cc5-b22a-50b6daac645e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:48.677773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:48.679634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:48.679925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:48.680974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] PATH_INFO: `/v3/projects/b3f95c3881524b21b30554109fdc3734` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:48.683481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f95c3881524b21b30554109fdc3734)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:48.685290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:48.784231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1022/2062] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:48 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:05:48.788932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794e5e8-925f-446d-bba2-1d0634018e1e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:48.801920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794e5e8-925f-446d-bba2-1d0634018e1e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:48.807620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644941a5-5809-41f1-a938-dc741f34aa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:48.808535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644941a5-5809-41f1-a938-dc741f34aa99 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:48.808889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644941a5-5809-41f1-a938-dc741f34aa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:48.809230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644941a5-5809-41f1-a938-dc741f34aa99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:48.815184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644941a5-5809-41f1-a938-dc741f34aa99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:48.818690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644941a5-5809-41f1-a938-dc741f34aa99 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:48.837199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1023/2063] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:48.847369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644941a5-5809-41f1-a938-dc741f34aa99 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:48.867082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644941a5-5809-41f1-a938-dc741f34aa99 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:48.873385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f95c3881524b21b30554109fdc3734', 'request_id': 'req-11d66e88-4307-443e-a41c-2982787e4778'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:48.883732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f95c3881524b21b30554109fdc3734', 'request_id': 'req-11d66e88-4307-443e-a41c-2982787e4778'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:48.932923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc07df-9376-4670-b9f0-071d1796d9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:48.933605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc07df-9376-4670-b9f0-071d1796d9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:48.933915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc07df-9376-4670-b9f0-071d1796d9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:48.934210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc07df-9376-4670-b9f0-071d1796d9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:48.934946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc07df-9376-4670-b9f0-071d1796d9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:48.937105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc07df-9376-4670-b9f0-071d1796d9f9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:48.947744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1024/2064] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:48.951851 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc07df-9376-4670-b9f0-071d1796d9f9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:48.962898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc07df-9376-4670-b9f0-071d1796d9f9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:48.969436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:48.970325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:48.970607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:48.970888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] PATH_INFO: `/v3/projects/535907d605c94446ac678131914ff4d6/users/fc0e66dcf2b04de5b7eca2ceaf279371/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:48.989310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f95c3881524b21b30554109fdc3734', 'request_id': 'req-11d66e88-4307-443e-a41c-2982787e4778'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:48.990806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=535907d605c94446ac678131914ff4d6, user_id=fc0e66dcf2b04de5b7eca2ceaf279371, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:48.995095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.015397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] The token cache is being invalidate because project b3f95c3881524b21b30554109fdc3734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:49.015397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.015397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.015397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1041/2065] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:48 2022] DELETE /identity/v3/projects/b3f95c3881524b21b30554109fdc3734 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:49.019887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:49.019887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.043439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d66e88-4307-443e-a41c-2982787e4778 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:49.048243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:49.049138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.049422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.049683 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] PATH_INFO: `/v3/projects/83615912b4384529aa09e0093c45e660` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.051083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=83615912b4384529aa09e0093c45e660)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:49.053100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.102758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1025/2066] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:48 2022] PUT /identity/v3/projects/535907d605c94446ac678131914ff4d6/users/fc0e66dcf2b04de5b7eca2ceaf279371/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:49.103721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:49.105464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.179444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65008e-226c-4166-970e-a4ddf95716c1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:49.192936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83615912b4384529aa09e0093c45e660', 'request_id': 'req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.199006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83615912b4384529aa09e0093c45e660', 'request_id': 'req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.247640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:49.248522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.248806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.249626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] PATH_INFO: `/v3/projects/68327a104e964ddca1304b29661f83e6` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.250926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68327a104e964ddca1304b29661f83e6)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:49.253047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.287350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83615912b4384529aa09e0093c45e660', 'request_id': 'req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.300066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] The token cache is being invalidate because project 83615912b4384529aa09e0093c45e660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:49.300365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.301034 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3ec05b7f-b9ea-46c8-bf54-dc9f9cd87a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.302600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1042/2067] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:49 2022] DELETE /identity/v3/projects/83615912b4384529aa09e0093c45e660 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:49.303984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee426dd-8726-453c-9e31-39cfedcfa741 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.304296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee426dd-8726-453c-9e31-39cfedcfa741 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.304568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee426dd-8726-453c-9e31-39cfedcfa741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.319628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:49.328646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bee426dd-8726-453c-9e31-39cfedcfa741 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:49.330001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bee426dd-8726-453c-9e31-39cfedcfa741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.338772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1043/2068] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:49.340370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c81c6a-d1b7-4de4-869c-2213e53c9b64 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.340774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c81c6a-d1b7-4de4-869c-2213e53c9b64 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.341051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c81c6a-d1b7-4de4-869c-2213e53c9b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.371902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-45c81c6a-d1b7-4de4-869c-2213e53c9b64 None None] MFA Rules not processed for user `fc0e66dcf2b04de5b7eca2ceaf279371`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:49.373197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-45c81c6a-d1b7-4de4-869c-2213e53c9b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.395047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68327a104e964ddca1304b29661f83e6', 'request_id': 'req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.413433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68327a104e964ddca1304b29661f83e6', 'request_id': 'req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.427310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68327a104e964ddca1304b29661f83e6', 'request_id': 'req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.461091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1044/2069] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:49.466490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40c2a-39b2-4adb-83a9-3ca6c9fb0885 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.466490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40c2a-39b2-4adb-83a9-3ca6c9fb0885 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.466490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40c2a-39b2-4adb-83a9-3ca6c9fb0885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.498296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-45b40c2a-39b2-4adb-83a9-3ca6c9fb0885 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:49.502533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-45b40c2a-39b2-4adb-83a9-3ca6c9fb0885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.534784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] The token cache is being invalidate because project 68327a104e964ddca1304b29661f83e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:49.535157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.535961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a2f666fd-7c17-4a6b-9c81-53c9c3c1b576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:49.537864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1026/2070] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:49 2022] DELETE /identity/v3/projects/68327a104e964ddca1304b29661f83e6 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:49.539199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf168466-9992-4535-bfcd-9e8f944ebfda None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.539614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf168466-9992-4535-bfcd-9e8f944ebfda None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.540198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf168466-9992-4535-bfcd-9e8f944ebfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.561365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-bf168466-9992-4535-bfcd-9e8f944ebfda None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:49.562817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bf168466-9992-4535-bfcd-9e8f944ebfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.605633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1045/2071] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:49.606503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b40c2a-39b2-4adb-83a9-3ca6c9fb0885 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:49.607841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-45b40c2a-39b2-4adb-83a9-3ca6c9fb0885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.653888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1027/2072] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:49.655848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63979467-3126-4c23-82ac-96fae68f31d5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.656137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63979467-3126-4c23-82ac-96fae68f31d5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.656413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63979467-3126-4c23-82ac-96fae68f31d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.678790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-63979467-3126-4c23-82ac-96fae68f31d5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:49.680068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-63979467-3126-4c23-82ac-96fae68f31d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.689215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b40c2a-39b2-4adb-83a9-3ca6c9fb0885 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:49.692383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1028/2073] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:49.693705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dfdf5-77fe-450e-909c-6006b8925935 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.694141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dfdf5-77fe-450e-909c-6006b8925935 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.694428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dfdf5-77fe-450e-909c-6006b8925935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.702359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d552ca-4edc-42ee-9864-71a3f9718aeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:49.703589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d552ca-4edc-42ee-9864-71a3f9718aeb service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.704080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d552ca-4edc-42ee-9864-71a3f9718aeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.704361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d552ca-4edc-42ee-9864-71a3f9718aeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.707383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e6d552ca-4edc-42ee-9864-71a3f9718aeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.718303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b85dfdf5-77fe-450e-909c-6006b8925935 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:49.719581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b85dfdf5-77fe-450e-909c-6006b8925935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.723384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d552ca-4edc-42ee-9864-71a3f9718aeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:49.724869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d552ca-4edc-42ee-9864-71a3f9718aeb service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.729969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1029/2074] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:49.730886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dfdf5-77fe-450e-909c-6006b8925935 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:49.738763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1046/2075] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:49.740141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f867-e18d-42b5-baf5-5db96f232978 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.740412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f867-e18d-42b5-baf5-5db96f232978 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.740848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f867-e18d-42b5-baf5-5db96f232978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.742206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dfdf5-77fe-450e-909c-6006b8925935 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:49.752040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:49.752726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.753012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.753290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.754035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:49.754344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:49.756006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.765992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4bf1f867-e18d-42b5-baf5-5db96f232978 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:49.768120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4bf1f867-e18d-42b5-baf5-5db96f232978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:49.793018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1030/2076] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:49.793018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:49.794050 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1047/2077] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:49.794879 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1f867-e18d-42b5-baf5-5db96f232978 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:49.807163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1f867-e18d-42b5-baf5-5db96f232978 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:49.808536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbf5c9-99b9-453c-ac30-c048acc49376 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:49.814137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:49.814907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.815489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.815962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.816969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:49.817374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:49.819103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.820592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ca645-d7ec-4519-8483-18ab4c21fcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:49.821596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ca645-d7ec-4519-8483-18ab4c21fcfe admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.822496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ca645-d7ec-4519-8483-18ab4c21fcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.824430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ca645-d7ec-4519-8483-18ab4c21fcfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.824430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ca645-d7ec-4519-8483-18ab4c21fcfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:49.826011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ca645-d7ec-4519-8483-18ab4c21fcfe admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.837463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1031/2078] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:49.837463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:49.852209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eadc8c-9768-430f-a6e6-c3b30339c91c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:49.857344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:49.859414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.859414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.859414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.859414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:49.859752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:49.861562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.876350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1032/2079] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:49.877186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:49.889331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254617fc-2631-4df9-9327-5109b87ad1a4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:49.894715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf655b-7f15-40e3-a9a4-cac71abb5254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:49.895392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf655b-7f15-40e3-a9a4-cac71abb5254 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:49.895670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf655b-7f15-40e3-a9a4-cac71abb5254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:49.895939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf655b-7f15-40e3-a9a4-cac71abb5254 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:49.896624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf655b-7f15-40e3-a9a4-cac71abb5254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:49.898500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf655b-7f15-40e3-a9a4-cac71abb5254 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:49.935074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1048/2080] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:49.935929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ca645-d7ec-4519-8483-18ab4c21fcfe admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.002897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ca645-d7ec-4519-8483-18ab4c21fcfe admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.008011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.008756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.009040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.009318 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.010047 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.012008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.052468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1033/2081] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:50.053309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf655b-7f15-40e3-a9a4-cac71abb5254 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.116490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf655b-7f15-40e3-a9a4-cac71abb5254 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.122193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb8b13-90f0-480f-a24a-372f54ccb9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.122876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8b13-90f0-480f-a24a-372f54ccb9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.123161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8b13-90f0-480f-a24a-372f54ccb9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.123472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8b13-90f0-480f-a24a-372f54ccb9a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.124195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb8b13-90f0-480f-a24a-372f54ccb9a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.126119 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb8b13-90f0-480f-a24a-372f54ccb9a7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.165560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1049/2082] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:49 2022] POST /identity/v3/projects => generated 385 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:50.166674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.168405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:50.228657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf4d29-ab3f-4310-84fc-e5d6593924ea admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.232573 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.233275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.233558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.233835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.235747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1034/2083] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:50 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:50.235931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:50.236755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb8b13-90f0-480f-a24a-372f54ccb9a7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.247965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb8b13-90f0-480f-a24a-372f54ccb9a7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.312837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.314350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.315106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d746f07-c9f5-4199-867f-d5c072916633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.315825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d746f07-c9f5-4199-867f-d5c072916633 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.316110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d746f07-c9f5-4199-867f-d5c072916633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.316388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d746f07-c9f5-4199-867f-d5c072916633 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.317089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d746f07-c9f5-4199-867f-d5c072916633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.319013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d746f07-c9f5-4199-867f-d5c072916633 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.324512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1050/2084] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:50.325337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.337531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed1c42-9967-475c-ac3b-c7497886ed1d service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.342662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6cd63-6ab2-41d7-94e9-9ca68cb00732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.343390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6cd63-6ab2-41d7-94e9-9ca68cb00732 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.343676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6cd63-6ab2-41d7-94e9-9ca68cb00732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.343950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6cd63-6ab2-41d7-94e9-9ca68cb00732 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.344672 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6cd63-6ab2-41d7-94e9-9ca68cb00732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.346669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6cd63-6ab2-41d7-94e9-9ca68cb00732 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.425405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1035/2085] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:50 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:50.426360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d746f07-c9f5-4199-867f-d5c072916633 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.438000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d746f07-c9f5-4199-867f-d5c072916633 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.447940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defff38-4f12-4ab4-a117-e00decf813ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.448631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defff38-4f12-4ab4-a117-e00decf813ad admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.448925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defff38-4f12-4ab4-a117-e00decf813ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.449208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defff38-4f12-4ab4-a117-e00decf813ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.453469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defff38-4f12-4ab4-a117-e00decf813ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.453469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defff38-4f12-4ab4-a117-e00decf813ad admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.460062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1051/2086] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:50 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:50.460943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6cd63-6ab2-41d7-94e9-9ca68cb00732 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.477700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1036/2087] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:50.477914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6cd63-6ab2-41d7-94e9-9ca68cb00732 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.478011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defff38-4f12-4ab4-a117-e00decf813ad admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.482861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f41ddd7-9545-456b-b5cc-ed227f32a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.484043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41ddd7-9545-456b-b5cc-ed227f32a06c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.484043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41ddd7-9545-456b-b5cc-ed227f32a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.484043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41ddd7-9545-456b-b5cc-ed227f32a06c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.484776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f41ddd7-9545-456b-b5cc-ed227f32a06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.486396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f41ddd7-9545-456b-b5cc-ed227f32a06c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.489164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defff38-4f12-4ab4-a117-e00decf813ad admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.494770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e40cd4-512c-4cf5-bebf-e4b477e7327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.495531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e40cd4-512c-4cf5-bebf-e4b477e7327e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.495827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e40cd4-512c-4cf5-bebf-e4b477e7327e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.496082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e40cd4-512c-4cf5-bebf-e4b477e7327e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.496813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e40cd4-512c-4cf5-bebf-e4b477e7327e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.497719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1052/2088] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:50.498478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e40cd4-512c-4cf5-bebf-e4b477e7327e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.498589 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f41ddd7-9545-456b-b5cc-ed227f32a06c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.513720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1037/2089] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:50.516474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f41ddd7-9545-456b-b5cc-ed227f32a06c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.516687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e40cd4-512c-4cf5-bebf-e4b477e7327e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.524183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16bef1-2508-45b1-9bee-c299b131dd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.525168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16bef1-2508-45b1-9bee-c299b131dd58 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.525572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16bef1-2508-45b1-9bee-c299b131dd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.525983 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16bef1-2508-45b1-9bee-c299b131dd58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.526286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e40cd4-512c-4cf5-bebf-e4b477e7327e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.527276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16bef1-2508-45b1-9bee-c299b131dd58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.530310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16bef1-2508-45b1-9bee-c299b131dd58 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.533420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd11f5a-9e3b-4ca0-b71e-1945ae12e4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.534115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11f5a-9e3b-4ca0-b71e-1945ae12e4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.534394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11f5a-9e3b-4ca0-b71e-1945ae12e4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.534674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11f5a-9e3b-4ca0-b71e-1945ae12e4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.535459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd11f5a-9e3b-4ca0-b71e-1945ae12e4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.537143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd11f5a-9e3b-4ca0-b71e-1945ae12e4f8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.549402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1038/2090] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:50.551325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1053/2091] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:50.552587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd11f5a-9e3b-4ca0-b71e-1945ae12e4f8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.554386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16bef1-2508-45b1-9bee-c299b131dd58 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.566994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16bef1-2508-45b1-9bee-c299b131dd58 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.571618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd11f5a-9e3b-4ca0-b71e-1945ae12e4f8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.572597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590e573-75d1-4f25-8acc-adce46c9ea90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.573678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e573-75d1-4f25-8acc-adce46c9ea90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.573976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e573-75d1-4f25-8acc-adce46c9ea90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.574263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e573-75d1-4f25-8acc-adce46c9ea90 admin admin] PATH_INFO: `/v3/projects/69e41f2cc08144a791e565996abd6041/users/ea22a6c26f0741fba1ad87c9118b84f9/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.579139 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2ef8a-7504-435c-a10f-d2a713a78d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.580199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2ef8a-7504-435c-a10f-d2a713a78d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.580542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2ef8a-7504-435c-a10f-d2a713a78d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.580908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2ef8a-7504-435c-a10f-d2a713a78d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.581801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2ef8a-7504-435c-a10f-d2a713a78d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.584625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2ef8a-7504-435c-a10f-d2a713a78d62 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.595857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590e573-75d1-4f25-8acc-adce46c9ea90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e41f2cc08144a791e565996abd6041, user_id=ea22a6c26f0741fba1ad87c9118b84f9, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.601311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590e573-75d1-4f25-8acc-adce46c9ea90 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.606607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1039/2092] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:50.608799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2ef8a-7504-435c-a10f-d2a713a78d62 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.621647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2ef8a-7504-435c-a10f-d2a713a78d62 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:50.628770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337220d-34c6-4d4e-9260-0ebd2c585622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:50.628770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337220d-34c6-4d4e-9260-0ebd2c585622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.629002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337220d-34c6-4d4e-9260-0ebd2c585622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.629081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337220d-34c6-4d4e-9260-0ebd2c585622 admin admin] PATH_INFO: `/v3/projects/3a1f4a376e804f409f452198d6e186c9/users/f08669c1c4e94d4187d15736462a4215/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.645848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337220d-34c6-4d4e-9260-0ebd2c585622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1f4a376e804f409f452198d6e186c9, user_id=f08669c1c4e94d4187d15736462a4215, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:50.648557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337220d-34c6-4d4e-9260-0ebd2c585622 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:50.804783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1040/2093] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:50 2022] PUT /identity/v3/projects/3a1f4a376e804f409f452198d6e186c9/users/f08669c1c4e94d4187d15736462a4215/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:50.811414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1054/2094] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:50 2022] PUT /identity/v3/projects/69e41f2cc08144a791e565996abd6041/users/ea22a6c26f0741fba1ad87c9118b84f9/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:50.817857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337220d-34c6-4d4e-9260-0ebd2c585622 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:50.821236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fb12-1aa6-4b40-923c-f05b4b72ec31 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:50.821236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fb12-1aa6-4b40-923c-f05b4b72ec31 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:50.821543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fb12-1aa6-4b40-923c-f05b4b72ec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:50.923517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:50.955534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-36b4fb12-1aa6-4b40-923c-f05b4b72ec31 None None] MFA Rules not processed for user `f08669c1c4e94d4187d15736462a4215`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:50.956535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-36b4fb12-1aa6-4b40-923c-f05b4b72ec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:50.965955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337220d-34c6-4d4e-9260-0ebd2c585622 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:51.053628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1055/2095] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:51.054944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce47f84-42cd-4c1e-8908-4efda3693378 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:51.055262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce47f84-42cd-4c1e-8908-4efda3693378 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:51.055560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce47f84-42cd-4c1e-8908-4efda3693378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:51.059610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d4cd5-b1d2-4bc1-9d2e-5bb523c63602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:51.060643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d4cd5-b1d2-4bc1-9d2e-5bb523c63602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:51.060968 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d4cd5-b1d2-4bc1-9d2e-5bb523c63602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:51.061318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d4cd5-b1d2-4bc1-9d2e-5bb523c63602 admin admin] PATH_INFO: `/v3/projects/822ca23f26ad45eb8357d061afb9a03a/users/0d157e92270546828c33aaa03247814b/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:51.087013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d4cd5-b1d2-4bc1-9d2e-5bb523c63602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=822ca23f26ad45eb8357d061afb9a03a, user_id=0d157e92270546828c33aaa03247814b, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:51.088169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bce47f84-42cd-4c1e-8908-4efda3693378 None None] MFA Rules not processed for user `ea22a6c26f0741fba1ad87c9118b84f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:51.089880 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bce47f84-42cd-4c1e-8908-4efda3693378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:51.092784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d4cd5-b1d2-4bc1-9d2e-5bb523c63602 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:51.174760 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1056/2096] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:51.175617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce47f84-42cd-4c1e-8908-4efda3693378 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:51.189144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce47f84-42cd-4c1e-8908-4efda3693378 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:51.196951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99a0ea-a273-417a-a343-a905ade6c306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:51.196951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a0ea-a273-417a-a343-a905ade6c306 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:51.196951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a0ea-a273-417a-a343-a905ade6c306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:51.196951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a0ea-a273-417a-a343-a905ade6c306 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:51.197343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99a0ea-a273-417a-a343-a905ade6c306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:51.198172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99a0ea-a273-417a-a343-a905ade6c306 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:51.303324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1041/2097] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:50 2022] PUT /identity/v3/projects/822ca23f26ad45eb8357d061afb9a03a/users/0d157e92270546828c33aaa03247814b/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:51.304930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d4cd5-b1d2-4bc1-9d2e-5bb523c63602 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:51.401249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d4cd5-b1d2-4bc1-9d2e-5bb523c63602 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:51.406798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a8c3e-dfd2-43e3-9431-e56b7989a3ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:51.407717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a8c3e-dfd2-43e3-9431-e56b7989a3ed service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:51.408005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a8c3e-dfd2-43e3-9431-e56b7989a3ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:51.408284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a8c3e-dfd2-43e3-9431-e56b7989a3ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:51.481882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a8c3e-dfd2-43e3-9431-e56b7989a3ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:51.483789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a8c3e-dfd2-43e3-9431-e56b7989a3ed service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:51.496360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1042/2098] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:51.499728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786687e3-3eea-491c-bbdc-aab5efadcd46 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:51.500857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786687e3-3eea-491c-bbdc-aab5efadcd46 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:51.501618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786687e3-3eea-491c-bbdc-aab5efadcd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:51.518121 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1057/2099] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:51 2022] POST /identity/v3/projects => generated 385 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:51.519078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99a0ea-a273-417a-a343-a905ade6c306 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:51.564505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-786687e3-3eea-491c-bbdc-aab5efadcd46 None None] MFA Rules not processed for user `0d157e92270546828c33aaa03247814b`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:51.566367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-786687e3-3eea-491c-bbdc-aab5efadcd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:51.619288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99a0ea-a273-417a-a343-a905ade6c306 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:51.629085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583620f1-18a7-455f-be58-1505887a9b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:51.630185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583620f1-18a7-455f-be58-1505887a9b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:51.634694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583620f1-18a7-455f-be58-1505887a9b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:51.634694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583620f1-18a7-455f-be58-1505887a9b1f admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:51.634694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583620f1-18a7-455f-be58-1505887a9b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:51.634694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583620f1-18a7-455f-be58-1505887a9b1f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:51.702269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1043/2100] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:52.006763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1058/2101] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:51 2022] POST /identity/v3/users => generated 299 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:52.009781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786687e3-3eea-491c-bbdc-aab5efadcd46 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:52.035483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786687e3-3eea-491c-bbdc-aab5efadcd46 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:52.043949 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48755a-e0a3-4ed1-9725-93d496f7e603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:52.045124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48755a-e0a3-4ed1-9725-93d496f7e603 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:52.045731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48755a-e0a3-4ed1-9725-93d496f7e603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:52.046234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48755a-e0a3-4ed1-9725-93d496f7e603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:52.047786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48755a-e0a3-4ed1-9725-93d496f7e603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:52.051127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48755a-e0a3-4ed1-9725-93d496f7e603 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:52.077007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1044/2102] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:52.079886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583620f1-18a7-455f-be58-1505887a9b1f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:52.095777 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583620f1-18a7-455f-be58-1505887a9b1f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:52.101373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c874861-3b06-44bb-b3fc-ca98ff6635a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:52.102334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c874861-3b06-44bb-b3fc-ca98ff6635a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:52.102828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c874861-3b06-44bb-b3fc-ca98ff6635a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:52.103053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c874861-3b06-44bb-b3fc-ca98ff6635a8 admin admin] PATH_INFO: `/v3/projects/92e6a6b0946b410d87127403ac8cfb32/users/224b348268894ac2807dc637a7a39574/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:52.123839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c874861-3b06-44bb-b3fc-ca98ff6635a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e6a6b0946b410d87127403ac8cfb32, user_id=224b348268894ac2807dc637a7a39574, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:52.130176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c874861-3b06-44bb-b3fc-ca98ff6635a8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:52.231947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:52.259337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1059/2103] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:52 2022] PUT /identity/v3/projects/92e6a6b0946b410d87127403ac8cfb32/users/224b348268894ac2807dc637a7a39574/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:52.263647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48755a-e0a3-4ed1-9725-93d496f7e603 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:52.274660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48755a-e0a3-4ed1-9725-93d496f7e603 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:52.338995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f38ef3-95fc-4b5d-b39c-fa84c662bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:52.339840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38ef3-95fc-4b5d-b39c-fa84c662bde1 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:52.339979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38ef3-95fc-4b5d-b39c-fa84c662bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:52.340263 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38ef3-95fc-4b5d-b39c-fa84c662bde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:52.341018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f38ef3-95fc-4b5d-b39c-fa84c662bde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:52.342751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f38ef3-95fc-4b5d-b39c-fa84c662bde1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:52.354478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1045/2104] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:52.359200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c874861-3b06-44bb-b3fc-ca98ff6635a8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:52.372346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c874861-3b06-44bb-b3fc-ca98ff6635a8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:52.378921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934e243-ffe1-49d6-9cd2-52f0afe5bedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:52.380020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934e243-ffe1-49d6-9cd2-52f0afe5bedd admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:52.380491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934e243-ffe1-49d6-9cd2-52f0afe5bedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:52.380870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934e243-ffe1-49d6-9cd2-52f0afe5bedd admin admin] PATH_INFO: `/v3/projects/92e6a6b0946b410d87127403ac8cfb32/users/224b348268894ac2807dc637a7a39574/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:52.383563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934e243-ffe1-49d6-9cd2-52f0afe5bedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e6a6b0946b410d87127403ac8cfb32, user_id=224b348268894ac2807dc637a7a39574, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:52.386597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934e243-ffe1-49d6-9cd2-52f0afe5bedd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:52.399358 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1060/2105] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:52 2022] PUT /identity/v3/projects/92e6a6b0946b410d87127403ac8cfb32/users/224b348268894ac2807dc637a7a39574/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:52.418892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cff33d-b7db-4b95-b630-a44c08ddab7d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:52.419630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cff33d-b7db-4b95-b630-a44c08ddab7d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:52.420229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cff33d-b7db-4b95-b630-a44c08ddab7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:52.472007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-98cff33d-b7db-4b95-b630-a44c08ddab7d None None] MFA Rules not processed for user `224b348268894ac2807dc637a7a39574`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:52.473451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-98cff33d-b7db-4b95-b630-a44c08ddab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:52.610246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1046/2106] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:52.698387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:52.729287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934e243-ffe1-49d6-9cd2-52f0afe5bedd admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:52.803789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934e243-ffe1-49d6-9cd2-52f0afe5bedd admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:52.808437 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:52.809329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:52.809626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:52.810183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:52.881572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cff33d-b7db-4b95-b630-a44c08ddab7d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:52.897870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cff33d-b7db-4b95-b630-a44c08ddab7d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:52.904459 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c0b4a-88a6-4a61-96a8-01dadd56861f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:52.905547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c0b4a-88a6-4a61-96a8-01dadd56861f service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:52.906327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c0b4a-88a6-4a61-96a8-01dadd56861f service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:52.907927 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c0b4a-88a6-4a61-96a8-01dadd56861f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:52.938978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:52.941885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:52.960963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1061/2107] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:52.997499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c0b4a-88a6-4a61-96a8-01dadd56861f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:52.999722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c0b4a-88a6-4a61-96a8-01dadd56861f service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:53.013771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1047/2108] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:53.517264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:54.390597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:54.906681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:55.323715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:55.325300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:55.420511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3dcb-cbbe-4f3b-9f12-67690918edf2 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:55.429217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:55.430787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:55.431595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:55.432100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] PATH_INFO: `/v3/users/fc0e66dcf2b04de5b7eca2ceaf279371` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:55.434345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc0e66dcf2b04de5b7eca2ceaf279371)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:55.439512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:55.600169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:55.722170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc0e66dcf2b04de5b7eca2ceaf279371'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:55.723032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc0e66dcf2b04de5b7eca2ceaf279371'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:55.737974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc0e66dcf2b04de5b7eca2ceaf279371'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:55.781351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc0e66dcf2b04de5b7eca2ceaf279371'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:55.795131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc0e66dcf2b04de5b7eca2ceaf279371'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:55.963263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1062/2109] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:55 2022] DELETE /identity/v3/users/fc0e66dcf2b04de5b7eca2ceaf279371 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:55.968084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932337d2-f059-4320-9203-e4338db5a9ca None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:55.968732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932337d2-f059-4320-9203-e4338db5a9ca None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:55.969732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932337d2-f059-4320-9203-e4338db5a9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:55.996969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:56.014863 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-932337d2-f059-4320-9203-e4338db5a9ca None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:56.018953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-932337d2-f059-4320-9203-e4338db5a9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:56.159011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1048/2110] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:56.172247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:56.175130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:56.269936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23950-d076-4d44-832d-f2c1ba79f7f5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:56.275001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54318db6-8958-4bbd-b876-cc6fcf4f865d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:56.275764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54318db6-8958-4bbd-b876-cc6fcf4f865d service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:56.276044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54318db6-8958-4bbd-b876-cc6fcf4f865d service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:56.276317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54318db6-8958-4bbd-b876-cc6fcf4f865d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:56.291279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54318db6-8958-4bbd-b876-cc6fcf4f865d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:56.292913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54318db6-8958-4bbd-b876-cc6fcf4f865d service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:56.312268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1063/2111] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:56.370958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1a89-2e94-4dcc-8ab8-dbd95b4da41f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:56.371677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1a89-2e94-4dcc-8ab8-dbd95b4da41f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:56.372129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1a89-2e94-4dcc-8ab8-dbd95b4da41f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:56.374848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1049/2112] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:05:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:05:56.387202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a5a-9650-4700-b4ad-3e15295b5983 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:56.387524 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a5a-9650-4700-b4ad-3e15295b5983 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:56.387802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a5a-9650-4700-b4ad-3e15295b5983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:56.410761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e4760a5a-9650-4700-b4ad-3e15295b5983 None None] MFA Rules not processed for user `f408d50bfc3a4366aabb1cdf86da7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:56.412092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e4760a5a-9650-4700-b4ad-3e15295b5983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:56.490015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1064/2113] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:05:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:56.503704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d1a89-2e94-4dcc-8ab8-dbd95b4da41f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:56.506180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e96d1a89-2e94-4dcc-8ab8-dbd95b4da41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:56.525319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d1a89-2e94-4dcc-8ab8-dbd95b4da41f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:56.543082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:56.546465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:56.546788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:56.547562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:56.578556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760a5a-9650-4700-b4ad-3e15295b5983 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:56.580730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e4760a5a-9650-4700-b4ad-3e15295b5983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:56.586039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:56.588047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:56.603534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1050/2114] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:05:56.605998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760a5a-9650-4700-b4ad-3e15295b5983 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:56.613300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:56.614582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:56.615023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:56.615482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] PATH_INFO: `/v3/users/8ab2da92b7e94592b80ef3d7852f51ae` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:56.617463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab2da92b7e94592b80ef3d7852f51ae)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:56.622047 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:56.827161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:56.829033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:56.843460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703fe5-d966-4b34-821e-ba8c8d1b947e service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:56.852339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:56.853609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:56.854119 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:56.858707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] PATH_INFO: `/v3/projects/535907d605c94446ac678131914ff4d6` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:56.858707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=535907d605c94446ac678131914ff4d6)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:56.858707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:56.876280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab2da92b7e94592b80ef3d7852f51ae'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:56.876954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab2da92b7e94592b80ef3d7852f51ae'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:56.877631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab2da92b7e94592b80ef3d7852f51ae'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:56.894076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab2da92b7e94592b80ef3d7852f51ae'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:56.903092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-41e86fff-afd1-4a4e-a2b9-0d58a522ad3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab2da92b7e94592b80ef3d7852f51ae'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.029169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1065/2115] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:56 2022] DELETE /identity/v3/users/8ab2da92b7e94592b80ef3d7852f51ae => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:57.065551 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535907d605c94446ac678131914ff4d6', 'request_id': 'req-c7e5ec75-ba38-4c91-938a-24bc08474d34'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.079963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535907d605c94446ac678131914ff4d6', 'request_id': 'req-c7e5ec75-ba38-4c91-938a-24bc08474d34'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.097548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535907d605c94446ac678131914ff4d6', 'request_id': 'req-c7e5ec75-ba38-4c91-938a-24bc08474d34'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.212486 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] The token cache is being invalidate because project 535907d605c94446ac678131914ff4d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:57.212889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.214230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c7e5ec75-ba38-4c91-938a-24bc08474d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.221927 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1051/2116] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:56 2022] DELETE /identity/v3/projects/535907d605c94446ac678131914ff4d6 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:05:57.224169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced62f4-93ed-40a3-8cc8-5ed0a20a8bb2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.228975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced62f4-93ed-40a3-8cc8-5ed0a20a8bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.228975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced62f4-93ed-40a3-8cc8-5ed0a20a8bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.258881 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cced62f4-93ed-40a3-8cc8-5ed0a20a8bb2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:57.261269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cced62f4-93ed-40a3-8cc8-5ed0a20a8bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:57.355869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1066/2117] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:57.367526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358124b-ed19-4359-9a0c-9f1be3b4420e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.368296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358124b-ed19-4359-9a0c-9f1be3b4420e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.368761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358124b-ed19-4359-9a0c-9f1be3b4420e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.392840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4358124b-ed19-4359-9a0c-9f1be3b4420e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:57.394101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4358124b-ed19-4359-9a0c-9f1be3b4420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:57.402732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1052/2118] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:57.415797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced62f4-93ed-40a3-8cc8-5ed0a20a8bb2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:57.424780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced62f4-93ed-40a3-8cc8-5ed0a20a8bb2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:57.432903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:57.434068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.434557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.435054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.436504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:57.437098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:57.440586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:57.460362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358124b-ed19-4359-9a0c-9f1be3b4420e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:57.462688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4358124b-ed19-4359-9a0c-9f1be3b4420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:57.463165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1067/2119] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:57.469567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:57.478449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358124b-ed19-4359-9a0c-9f1be3b4420e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:57.483506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b67b6-0c26-4ee8-b85f-da000a354b66 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:57.484718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:57.485634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.486237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.486681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] PATH_INFO: `/v3/projects/cb56c390ed7649479455d6dc3904deb0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.488246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb56c390ed7649479455d6dc3904deb0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:57.489599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed26f9-f002-4650-adc8-4eb5169a350e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:57.490327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed26f9-f002-4650-adc8-4eb5169a350e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.490606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed26f9-f002-4650-adc8-4eb5169a350e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.499579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:57.500142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed26f9-f002-4650-adc8-4eb5169a350e admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.500142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed26f9-f002-4650-adc8-4eb5169a350e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:57.500142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed26f9-f002-4650-adc8-4eb5169a350e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:57.650919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1068/2120] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:57 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:05:57.654657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed26f9-f002-4650-adc8-4eb5169a350e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:57.711960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb56c390ed7649479455d6dc3904deb0', 'request_id': 'req-2006009d-7f4a-4956-b56b-492293e166f4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.719046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb56c390ed7649479455d6dc3904deb0', 'request_id': 'req-2006009d-7f4a-4956-b56b-492293e166f4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.725833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb56c390ed7649479455d6dc3904deb0', 'request_id': 'req-2006009d-7f4a-4956-b56b-492293e166f4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.740675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed26f9-f002-4650-adc8-4eb5169a350e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:57.799966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:57.822669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:57.823876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.824301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.824845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.826146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:57.830864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:57.848728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] The token cache is being invalidate because project cb56c390ed7649479455d6dc3904deb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:05:57.849344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.850427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2006009d-7f4a-4956-b56b-492293e166f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:57.852455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1053/2121] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:57 2022] DELETE /identity/v3/projects/cb56c390ed7649479455d6dc3904deb0 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:57.858253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eddd01-a535-4360-93f5-4373f6aae433 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.858607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eddd01-a535-4360-93f5-4373f6aae433 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.859019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eddd01-a535-4360-93f5-4373f6aae433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.886571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d8eddd01-a535-4360-93f5-4373f6aae433 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:57.887848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d8eddd01-a535-4360-93f5-4373f6aae433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:57.896966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1054/2122] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:57.909042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34911-ac05-46c4-ac40-bfe3a16cb94c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.909451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34911-ac05-46c4-ac40-bfe3a16cb94c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.909817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34911-ac05-46c4-ac40-bfe3a16cb94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.930500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0bf34911-ac05-46c4-ac40-bfe3a16cb94c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:57.931846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0bf34911-ac05-46c4-ac40-bfe3a16cb94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:57.941945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1055/2123] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:05:57.944070 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1069/2124] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:57 2022] POST /identity/v3/users => generated 288 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:05:57.951039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf34911-ac05-46c4-ac40-bfe3a16cb94c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:57.951203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:57.955112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:57.975546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf34911-ac05-46c4-ac40-bfe3a16cb94c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:57.977064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:57.977705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81223-49c0-46ce-9244-ef1234fe44a7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:57.978201 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.978701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.979192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.980717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:05:57.981237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:57.984656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:57.988757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08cb44-a641-4773-b162-e524247ed8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:57.990020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08cb44-a641-4773-b162-e524247ed8ef admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:57.990480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08cb44-a641-4773-b162-e524247ed8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:57.990928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08cb44-a641-4773-b162-e524247ed8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:57.992379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08cb44-a641-4773-b162-e524247ed8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:57.995189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08cb44-a641-4773-b162-e524247ed8ef admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.006577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1056/2125] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:05:58.008226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.012776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1070/2126] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:58.013882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:58.015687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08cb44-a641-4773-b162-e524247ed8ef admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.033487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08cb44-a641-4773-b162-e524247ed8ef admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.039688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2011b2-8219-4ee8-851b-a6ceb0206ae2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.041370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727d394-7a32-42c0-a588-35a74e492604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.042440 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727d394-7a32-42c0-a588-35a74e492604 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.042913 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727d394-7a32-42c0-a588-35a74e492604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.043830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727d394-7a32-42c0-a588-35a74e492604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.044627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727d394-7a32-42c0-a588-35a74e492604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.047712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.048156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727d394-7a32-42c0-a588-35a74e492604 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.049095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.049545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.050094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] PATH_INFO: `/v3/users/c34fbaf3fd7c415e81aa1d501ca99e25` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.052270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34fbaf3fd7c415e81aa1d501ca99e25)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.056218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.082207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:58.213335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1071/2127] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:58 2022] POST /identity/v3/projects => generated 403 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:58.214206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727d394-7a32-42c0-a588-35a74e492604 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.289802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727d394-7a32-42c0-a588-35a74e492604 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.294944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a3e39-1260-4d71-81b3-7134a0d6e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.296154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a3e39-1260-4d71-81b3-7134a0d6e47d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.296436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a3e39-1260-4d71-81b3-7134a0d6e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.296710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a3e39-1260-4d71-81b3-7134a0d6e47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.297493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a3e39-1260-4d71-81b3-7134a0d6e47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.299125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a3e39-1260-4d71-81b3-7134a0d6e47d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.312091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1072/2128] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:58.313537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a3e39-1260-4d71-81b3-7134a0d6e47d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.328365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a3e39-1260-4d71-81b3-7134a0d6e47d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.331955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34fbaf3fd7c415e81aa1d501ca99e25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:58.332647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34fbaf3fd7c415e81aa1d501ca99e25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:58.333241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d0ac8-3969-4617-b68c-e187325d72c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.333923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d0ac8-3969-4617-b68c-e187325d72c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.334201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d0ac8-3969-4617-b68c-e187325d72c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.334473 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d0ac8-3969-4617-b68c-e187325d72c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.335197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d0ac8-3969-4617-b68c-e187325d72c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.337173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d0ac8-3969-4617-b68c-e187325d72c0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.430973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34fbaf3fd7c415e81aa1d501ca99e25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:58.445208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34fbaf3fd7c415e81aa1d501ca99e25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:58.446377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34fbaf3fd7c415e81aa1d501ca99e25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:58.474085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1057/2129] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:05:58 2022] DELETE /identity/v3/users/c34fbaf3fd7c415e81aa1d501ca99e25 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:58.475532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.477899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:58.492162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1073/2130] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:58 2022] POST /identity/v3/users => generated 302 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:58.493850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc093bf-aca5-4a88-b983-0dee63ad7a0f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.497351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc093bf-aca5-4a88-b983-0dee63ad7a0f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.497351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc093bf-aca5-4a88-b983-0dee63ad7a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.518141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2fc093bf-aca5-4a88-b983-0dee63ad7a0f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:58.535690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2fc093bf-aca5-4a88-b983-0dee63ad7a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:58.586647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e221c1e-af79-4c2b-90d8-02c630003de7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.593697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc686a-d2b3-43bf-8ab5-2fe8db5b44e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.595532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc686a-d2b3-43bf-8ab5-2fe8db5b44e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.596100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc686a-d2b3-43bf-8ab5-2fe8db5b44e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.596100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc686a-d2b3-43bf-8ab5-2fe8db5b44e8 admin admin] PATH_INFO: `/v3/projects/12c54764cae34c12b35a6939f96aad5c/users/d6515163518749d3bbe6e3eeda0a4019/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.598371 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1074/2131] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:58.599486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc093bf-aca5-4a88-b983-0dee63ad7a0f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.601087 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2fc093bf-aca5-4a88-b983-0dee63ad7a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:58.617394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc093bf-aca5-4a88-b983-0dee63ad7a0f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.618530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc686a-d2b3-43bf-8ab5-2fe8db5b44e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c54764cae34c12b35a6939f96aad5c, user_id=d6515163518749d3bbe6e3eeda0a4019, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.621296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc686a-d2b3-43bf-8ab5-2fe8db5b44e8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.622430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.623216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.623533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.624478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.625554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.626869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.638028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1075/2132] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:58.638914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.641794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:58.705941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1058/2133] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:58 2022] PUT /identity/v3/projects/12c54764cae34c12b35a6939f96aad5c/users/d6515163518749d3bbe6e3eeda0a4019/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:58.707036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc686a-d2b3-43bf-8ab5-2fe8db5b44e8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.711544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb3f03-e51d-40b5-bf3b-ae22ffaf3e1a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.778042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc686a-d2b3-43bf-8ab5-2fe8db5b44e8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.783363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b66fd-1c8d-4e25-bd21-e808a5cfbd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.784067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b66fd-1c8d-4e25-bd21-e808a5cfbd31 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.784352 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b66fd-1c8d-4e25-bd21-e808a5cfbd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.784634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b66fd-1c8d-4e25-bd21-e808a5cfbd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.785394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b66fd-1c8d-4e25-bd21-e808a5cfbd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.787086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b66fd-1c8d-4e25-bd21-e808a5cfbd31 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.793105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc644-1979-4db0-9e52-b799f4fdd2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.802395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc644-1979-4db0-9e52-b799f4fdd2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.802395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc644-1979-4db0-9e52-b799f4fdd2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.802395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc644-1979-4db0-9e52-b799f4fdd2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.802864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1059/2134] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:05:58.802864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86838246-5424-4921-9252-3a35acaa4129 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.802864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86838246-5424-4921-9252-3a35acaa4129 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.802864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86838246-5424-4921-9252-3a35acaa4129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.813068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafc644-1979-4db0-9e52-b799f4fdd2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.814635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafc644-1979-4db0-9e52-b799f4fdd2f6 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.825982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1076/2135] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:58.826724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc644-1979-4db0-9e52-b799f4fdd2f6 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.837960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc644-1979-4db0-9e52-b799f4fdd2f6 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:58.838230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-86838246-5424-4921-9252-3a35acaa4129 None None] MFA Rules not processed for user `d6515163518749d3bbe6e3eeda0a4019`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:58.839661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-86838246-5424-4921-9252-3a35acaa4129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:58.843520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e324e2-8700-4951-aca4-0db879d3b8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:58.844455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e324e2-8700-4951-aca4-0db879d3b8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.844740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e324e2-8700-4951-aca4-0db879d3b8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.845010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e324e2-8700-4951-aca4-0db879d3b8a0 admin admin] PATH_INFO: `/v3/projects/ac0d2f89fe984e3cadd28fe1a781e412/users/0cc70d2b52cd4ac5b6de62e1718ea470/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:58.863283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e324e2-8700-4951-aca4-0db879d3b8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0d2f89fe984e3cadd28fe1a781e412, user_id=0cc70d2b52cd4ac5b6de62e1718ea470, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:58.866148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e324e2-8700-4951-aca4-0db879d3b8a0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:58.948657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1060/2136] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:58.969079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1077/2137] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:58 2022] PUT /identity/v3/projects/ac0d2f89fe984e3cadd28fe1a781e412/users/0cc70d2b52cd4ac5b6de62e1718ea470/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:05:58.969398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86838246-5424-4921-9252-3a35acaa4129 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:58.969398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-86838246-5424-4921-9252-3a35acaa4129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:58.973890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5ca06-92ed-44ef-9165-97052df4bfaf None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:58.974328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5ca06-92ed-44ef-9165-97052df4bfaf None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:58.974729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5ca06-92ed-44ef-9165-97052df4bfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:59.008280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c3e5ca06-92ed-44ef-9165-97052df4bfaf None None] MFA Rules not processed for user `0cc70d2b52cd4ac5b6de62e1718ea470`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:05:59.010279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c3e5ca06-92ed-44ef-9165-97052df4bfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:59.055572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86838246-5424-4921-9252-3a35acaa4129 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:59.080647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:05:59.158437 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1078/2138] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:05:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:05:59.166743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a13d5-d080-48d8-b6c9-52eb90cd751b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:59.167738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a13d5-d080-48d8-b6c9-52eb90cd751b service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:59.168197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a13d5-d080-48d8-b6c9-52eb90cd751b service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:59.168477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a13d5-d080-48d8-b6c9-52eb90cd751b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:59.171471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5ca06-92ed-44ef-9165-97052df4bfaf None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:59.273010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a13d5-d080-48d8-b6c9-52eb90cd751b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:59.274559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a13d5-d080-48d8-b6c9-52eb90cd751b service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:59.286983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1061/2139] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:05:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:05:59.300968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5ca06-92ed-44ef-9165-97052df4bfaf None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:59.306573 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:59.307272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:59.307560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:59.307834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:59.308612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:59.310797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:59.459167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1079/2140] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:05:59 2022] POST /identity/v3/projects => generated 403 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:59.463615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a13d5-d080-48d8-b6c9-52eb90cd751b service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:59.477253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a13d5-d080-48d8-b6c9-52eb90cd751b service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:59.488309 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:59.492154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:05:59.593836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc9ef2-e79f-4cc0-b426-def7fa5b74e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:59.594720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc9ef2-e79f-4cc0-b426-def7fa5b74e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:59.595761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc9ef2-e79f-4cc0-b426-def7fa5b74e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:59.595761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc9ef2-e79f-4cc0-b426-def7fa5b74e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:59.596884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc9ef2-e79f-4cc0-b426-def7fa5b74e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:59.599613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc9ef2-e79f-4cc0-b426-def7fa5b74e5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:59.601397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a11f8-963e-403c-a87f-9cd2ca3f7c09 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:59.609353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:59.610394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:59.610678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:59.610944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] PATH_INFO: `/v3/projects/e06ba22c68ac42e6b27270ff299462ff` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:59.612623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06ba22c68ac42e6b27270ff299462ff)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:59.615169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:59.825687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1062/2141] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:05:59 2022] POST /identity/v3/users => generated 308 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:05:59.838783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc9ef2-e79f-4cc0-b426-def7fa5b74e5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:59.849861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc9ef2-e79f-4cc0-b426-def7fa5b74e5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:59.854696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958cbc-3430-4d49-b9b1-dd0f4e710372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:05:59.855424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958cbc-3430-4d49-b9b1-dd0f4e710372 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:05:59.855713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958cbc-3430-4d49-b9b1-dd0f4e710372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:05:59.855986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958cbc-3430-4d49-b9b1-dd0f4e710372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:05:59.856762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91958cbc-3430-4d49-b9b1-dd0f4e710372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:05:59.859769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91958cbc-3430-4d49-b9b1-dd0f4e710372 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:05:59.886555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1063/2142] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:05:59.890905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958cbc-3430-4d49-b9b1-dd0f4e710372 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:05:59.893619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06ba22c68ac42e6b27270ff299462ff', 'request_id': 'req-512932e5-bfc6-4768-b91d-d491c44aad88'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:59.903593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06ba22c68ac42e6b27270ff299462ff', 'request_id': 'req-512932e5-bfc6-4768-b91d-d491c44aad88'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:05:59.908490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958cbc-3430-4d49-b9b1-dd0f4e710372 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:05:59.950434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:00.010170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:00.011791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:00.012310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:00.012819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] PATH_INFO: `/v3/projects/1cc6cb68cd664bd888b13f2a9864d868/users/508fab5e46e64100b5924d03fcf88c8d/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:00.037549 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc6cb68cd664bd888b13f2a9864d868, user_id=508fab5e46e64100b5924d03fcf88c8d, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:00.040756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:00.042033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06ba22c68ac42e6b27270ff299462ff', 'request_id': 'req-512932e5-bfc6-4768-b91d-d491c44aad88'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:00.056148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] The token cache is being invalidate because project e06ba22c68ac42e6b27270ff299462ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:00.056442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:00.057127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:00.058732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1080/2143] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:05:59 2022] DELETE /identity/v3/projects/e06ba22c68ac42e6b27270ff299462ff => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:00.063177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:00.064742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:00.078462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512932e5-bfc6-4768-b91d-d491c44aad88 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:00.083414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:00.084351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:00.084639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:00.084920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] PATH_INFO: `/v3/users/62c13b6c4c934fa5b2ed20bc3a521633` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:00.086243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c13b6c4c934fa5b2ed20bc3a521633)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:00.088290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:00.175966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1064/2144] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:05:59 2022] PUT /identity/v3/projects/1cc6cb68cd664bd888b13f2a9864d868/users/508fab5e46e64100b5924d03fcf88c8d/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:00.180243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:00.182073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:00.271024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e437e-e768-46a0-993b-b27673e84259 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:00.282808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceddad-1f60-4dea-8c6c-ab579345a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:00.283836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceddad-1f60-4dea-8c6c-ab579345a094 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:00.284120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceddad-1f60-4dea-8c6c-ab579345a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:00.284490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceddad-1f60-4dea-8c6c-ab579345a094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:00.285541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ceddad-1f60-4dea-8c6c-ab579345a094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:00.287562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ceddad-1f60-4dea-8c6c-ab579345a094 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:00.303187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1065/2145] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:00.309286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceddad-1f60-4dea-8c6c-ab579345a094 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:00.322445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceddad-1f60-4dea-8c6c-ab579345a094 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:00.332356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf5962-2cc9-4c15-9c10-3490160d0ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:00.336237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5962-2cc9-4c15-9c10-3490160d0ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:00.336693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5962-2cc9-4c15-9c10-3490160d0ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:00.337162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5962-2cc9-4c15-9c10-3490160d0ddf admin admin] PATH_INFO: `/v3/projects/1cc6cb68cd664bd888b13f2a9864d868/users/508fab5e46e64100b5924d03fcf88c8d/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:00.341245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf5962-2cc9-4c15-9c10-3490160d0ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc6cb68cd664bd888b13f2a9864d868, user_id=508fab5e46e64100b5924d03fcf88c8d, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:00.346234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf5962-2cc9-4c15-9c10-3490160d0ddf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:00.358495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c13b6c4c934fa5b2ed20bc3a521633'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:00.359592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c13b6c4c934fa5b2ed20bc3a521633'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:00.366288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c13b6c4c934fa5b2ed20bc3a521633'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:00.374071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1066/2146] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:00 2022] PUT /identity/v3/projects/1cc6cb68cd664bd888b13f2a9864d868/users/508fab5e46e64100b5924d03fcf88c8d/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:00.386209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645bc16-b14a-43b5-863d-5a31546b66e9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:00.386695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645bc16-b14a-43b5-863d-5a31546b66e9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:00.386853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645bc16-b14a-43b5-863d-5a31546b66e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:00.392152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c13b6c4c934fa5b2ed20bc3a521633'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:00.410707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c13b6c4c934fa5b2ed20bc3a521633'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:00.433253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e645bc16-b14a-43b5-863d-5a31546b66e9 None None] MFA Rules not processed for user `508fab5e46e64100b5924d03fcf88c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:00.434709 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e645bc16-b14a-43b5-863d-5a31546b66e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:00.515576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1081/2147] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:00 2022] DELETE /identity/v3/users/62c13b6c4c934fa5b2ed20bc3a521633 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:00.554791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1067/2148] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:00.837765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:00.917835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645bc16-b14a-43b5-863d-5a31546b66e9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:00.918002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:00.919681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:00.920561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e645bc16-b14a-43b5-863d-5a31546b66e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:00.985815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:01.010610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645bc16-b14a-43b5-863d-5a31546b66e9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:01.023671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ea3e9-9a80-4b92-a7ef-846762b89400 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:01.027994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:01.029368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:01.030111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:01.035043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:01.124465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:01.135963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:01.135963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:01.135963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] PATH_INFO: `/v3/users/ea22a6c26f0741fba1ad87c9118b84f9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:01.135963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea22a6c26f0741fba1ad87c9118b84f9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:01.135963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:01.176224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:01.179242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:01.197009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1082/2149] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:00 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:01.197987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:01.200055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:01.215663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e98fd9-935d-4427-a773-6a381d744323 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:01.222376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:01.223972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:01.224602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:01.225127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] PATH_INFO: `/v3/projects/c1b13a1d309544a2acb650c89b4461dc` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:01.227962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b13a1d309544a2acb650c89b4461dc)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:01.236399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:01.420069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea22a6c26f0741fba1ad87c9118b84f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.421612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea22a6c26f0741fba1ad87c9118b84f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.450836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b13a1d309544a2acb650c89b4461dc', 'request_id': 'req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.457883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b13a1d309544a2acb650c89b4461dc', 'request_id': 'req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.533580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea22a6c26f0741fba1ad87c9118b84f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.542583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea22a6c26f0741fba1ad87c9118b84f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.543734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea22a6c26f0741fba1ad87c9118b84f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.568819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1068/2150] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:00 2022] DELETE /identity/v3/users/ea22a6c26f0741fba1ad87c9118b84f9 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:01.570351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:01.572952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:01.665069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b13a1d309544a2acb650c89b4461dc', 'request_id': 'req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.670618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd12b64-8aba-4285-9c0c-b41503e19f0b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:01.679095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:01.680357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] The token cache is being invalidate because project c1b13a1d309544a2acb650c89b4461dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:01.680711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.681336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:01.681655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:01.681767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c891dbaf-38e9-4405-8dfc-ef0f508c6d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.681947 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] PATH_INFO: `/v3/users/0cc70d2b52cd4ac5b6de62e1718ea470` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:01.683621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1083/2151] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:01 2022] DELETE /identity/v3/projects/c1b13a1d309544a2acb650c89b4461dc => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:01.685303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8203b4d-6967-4346-8909-cc9f65cff5f5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:01.685645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8203b4d-6967-4346-8909-cc9f65cff5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:01.686768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8203b4d-6967-4346-8909-cc9f65cff5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:01.688012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc70d2b52cd4ac5b6de62e1718ea470)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:01.690454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:01.717583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f8203b4d-6967-4346-8909-cc9f65cff5f5 None None] MFA Rules not processed for user `d6515163518749d3bbe6e3eeda0a4019`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:01.719076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f8203b4d-6967-4346-8909-cc9f65cff5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:01.785472 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1084/2152] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:01.786840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24769f1-fb80-49b0-a3fe-606f89df37f0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:01.787112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24769f1-fb80-49b0-a3fe-606f89df37f0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:01.787428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24769f1-fb80-49b0-a3fe-606f89df37f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:01.815751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c24769f1-fb80-49b0-a3fe-606f89df37f0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:01.818085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c24769f1-fb80-49b0-a3fe-606f89df37f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:01.832546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1085/2153] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:01.833850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24769f1-fb80-49b0-a3fe-606f89df37f0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:01.837231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c24769f1-fb80-49b0-a3fe-606f89df37f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:01.857605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24769f1-fb80-49b0-a3fe-606f89df37f0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:01.862730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:01.865509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:01.865784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:01.871533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] PATH_INFO: `/v3/users/5a2ccb41890146978ddd76f3077dbf7e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:01.873032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2ccb41890146978ddd76f3077dbf7e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:01.875036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:01.922976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc70d2b52cd4ac5b6de62e1718ea470'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:01.923616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc70d2b52cd4ac5b6de62e1718ea470'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.087456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc70d2b52cd4ac5b6de62e1718ea470'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.097912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc70d2b52cd4ac5b6de62e1718ea470'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.097912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc70d2b52cd4ac5b6de62e1718ea470'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.120689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1069/2154] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:01 2022] DELETE /identity/v3/users/0cc70d2b52cd4ac5b6de62e1718ea470 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:02.124194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:02.124194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:02.180082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:02.197335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2ccb41890146978ddd76f3077dbf7e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.198808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2ccb41890146978ddd76f3077dbf7e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.200040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2ccb41890146978ddd76f3077dbf7e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.228923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2ccb41890146978ddd76f3077dbf7e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.245287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c19051ab-0c5f-49a8-9888-2356b84d0571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2ccb41890146978ddd76f3077dbf7e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:02.275091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88093902-de81-4a4f-baab-950b161af7ca admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:02.283677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:02.288677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:02.292513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:02.292513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:02.292513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:02.294830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:02.391909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1086/2155] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:01 2022] DELETE /identity/v3/users/5a2ccb41890146978ddd76f3077dbf7e => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:02.393870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660a022-3af4-4da4-a978-3ece18c2d526 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:02.394681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660a022-3af4-4da4-a978-3ece18c2d526 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:02.395152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660a022-3af4-4da4-a978-3ece18c2d526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:02.440498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:02.442182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:02.456695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1070/2156] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:02.457791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:02.459710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:02.473086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f660a022-3af4-4da4-a978-3ece18c2d526 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:02.479314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f660a022-3af4-4da4-a978-3ece18c2d526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:02.602191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f9ef0-691a-4e29-bc43-97cc4268735a service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:02.664406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1087/2157] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:02.676789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660a022-3af4-4da4-a978-3ece18c2d526 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:02.678459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f660a022-3af4-4da4-a978-3ece18c2d526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:02.690916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:02.692151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:02.692625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:02.693082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:02.695868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:02.813222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:02.815034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:02.831912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1071/2158] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:02.842906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660a022-3af4-4da4-a978-3ece18c2d526 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:02.847836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:02.848592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:02.848943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:02.849297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:02.861682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:02.863480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:02.879809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1088/2159] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:06:03.270335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:03.272079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:03.288850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d48a9-7968-4b22-a888-a99242d9db47 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:03.294906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:03.296507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:03.296849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:03.297133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] PATH_INFO: `/v3/projects/1b07c800ff9243b5bc4a4f52ff0ffc28` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:03.298683 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b07c800ff9243b5bc4a4f52ff0ffc28)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:03.300996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:03.477889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b07c800ff9243b5bc4a4f52ff0ffc28', 'request_id': 'req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.486534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b07c800ff9243b5bc4a4f52ff0ffc28', 'request_id': 'req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.494584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b07c800ff9243b5bc4a4f52ff0ffc28', 'request_id': 'req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.505635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:03.507313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:03.529157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578f74-4659-403f-a962-a499a8a0f1e4 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:03.596339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:03.619802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:03.620836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:03.621162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:03.621536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] PATH_INFO: `/v3/projects/69e41f2cc08144a791e565996abd6041` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:03.622986 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e41f2cc08144a791e565996abd6041)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:03.625311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:03.644280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] The token cache is being invalidate because project 1b07c800ff9243b5bc4a4f52ff0ffc28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:03.645149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.645853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.649338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1072/2160] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:03 2022] DELETE /identity/v3/projects/1b07c800ff9243b5bc4a4f52ff0ffc28 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:03.650606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:03.653414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:03.680279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d34d6d-5a71-4ede-93b8-be52bfb01ded admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:03.694393 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:03.695436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:03.695746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:03.696063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] PATH_INFO: `/v3/projects/ac0d2f89fe984e3cadd28fe1a781e412` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:03.697574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0d2f89fe984e3cadd28fe1a781e412)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:03.699615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:03.846133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0d2f89fe984e3cadd28fe1a781e412', 'request_id': 'req-63553338-b5f8-4b4b-b72b-371136383ec5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.846627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e41f2cc08144a791e565996abd6041', 'request_id': 'req-70d06019-f617-48e6-8549-7a5c0e96bd13'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.855994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e41f2cc08144a791e565996abd6041', 'request_id': 'req-70d06019-f617-48e6-8549-7a5c0e96bd13'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.876001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0d2f89fe984e3cadd28fe1a781e412', 'request_id': 'req-63553338-b5f8-4b4b-b72b-371136383ec5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:03.892197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0d2f89fe984e3cadd28fe1a781e412', 'request_id': 'req-63553338-b5f8-4b4b-b72b-371136383ec5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.013491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e41f2cc08144a791e565996abd6041', 'request_id': 'req-70d06019-f617-48e6-8549-7a5c0e96bd13'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.026048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] The token cache is being invalidate because project 69e41f2cc08144a791e565996abd6041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:04.026340 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.027023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-70d06019-f617-48e6-8549-7a5c0e96bd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.028676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1089/2161] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:03 2022] DELETE /identity/v3/projects/69e41f2cc08144a791e565996abd6041 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:04.030028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95078f65-7576-442e-9bdf-e515c7f238cb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.030503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95078f65-7576-442e-9bdf-e515c7f238cb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.030928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95078f65-7576-442e-9bdf-e515c7f238cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.065344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-95078f65-7576-442e-9bdf-e515c7f238cb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:04.067458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-95078f65-7576-442e-9bdf-e515c7f238cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:04.155268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1090/2162] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:04.157077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cea33e-bed9-4f3c-b7d4-d7bf5d0e4525 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.157445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cea33e-bed9-4f3c-b7d4-d7bf5d0e4525 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.161718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cea33e-bed9-4f3c-b7d4-d7bf5d0e4525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.181088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-52cea33e-bed9-4f3c-b7d4-d7bf5d0e4525 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:04.182326 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-52cea33e-bed9-4f3c-b7d4-d7bf5d0e4525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:04.193007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:04.196714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] The token cache is being invalidate because project ac0d2f89fe984e3cadd28fe1a781e412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:04.197224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.198152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-63553338-b5f8-4b4b-b72b-371136383ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.201171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1073/2163] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:03 2022] DELETE /identity/v3/projects/ac0d2f89fe984e3cadd28fe1a781e412 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:04.206095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fb78b-0b88-4866-b040-9ce8d8278023 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.206095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fb78b-0b88-4866-b040-9ce8d8278023 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.206095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fb78b-0b88-4866-b040-9ce8d8278023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.213392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1091/2164] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:04.215328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cea33e-bed9-4f3c-b7d4-d7bf5d0e4525 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:04.217543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-52cea33e-bed9-4f3c-b7d4-d7bf5d0e4525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:04.240348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cea33e-bed9-4f3c-b7d4-d7bf5d0e4525 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:04.249878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:04.251335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.251769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.252224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] PATH_INFO: `/v3/users/508fab5e46e64100b5924d03fcf88c8d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.254321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508fab5e46e64100b5924d03fcf88c8d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:04.254542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ea7fb78b-0b88-4866-b040-9ce8d8278023 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:04.257087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ea7fb78b-0b88-4866-b040-9ce8d8278023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:04.257875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:04.284225 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1074/2165] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:04.287167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88ff2-0cb7-4be4-8ce8-c620f8c9a78d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.289533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88ff2-0cb7-4be4-8ce8-c620f8c9a78d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.290363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88ff2-0cb7-4be4-8ce8-c620f8c9a78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.329120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-64b88ff2-0cb7-4be4-8ce8-c620f8c9a78d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:04.330361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-64b88ff2-0cb7-4be4-8ce8-c620f8c9a78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:04.339124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1075/2166] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:04.340023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88ff2-0cb7-4be4-8ce8-c620f8c9a78d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:04.354273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88ff2-0cb7-4be4-8ce8-c620f8c9a78d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:04.360722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:04.361544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.361874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.362345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.363351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:04.363753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:04.365532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:04.381021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1076/2167] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:04.382524 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:04.402754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc1103-bf35-4897-8534-e4f8f5223bf5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:04.411102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:04.412214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.417564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.417564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.417564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:04.417564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:04.418130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:04.439428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1077/2168] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:04.441095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:04.459601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0b4d7-3108-4297-a1fa-7a5f9c82cd0e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:04.464630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:04.465299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.465581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.465858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.466608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:04.469296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:04.572437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1078/2169] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:04 2022] POST /identity/v3/projects => generated 409 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:04.573471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508fab5e46e64100b5924d03fcf88c8d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.574258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:04.574403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508fab5e46e64100b5924d03fcf88c8d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.575102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508fab5e46e64100b5924d03fcf88c8d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.577696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:04.593831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508fab5e46e64100b5924d03fcf88c8d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.606897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508fab5e46e64100b5924d03fcf88c8d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:04.666645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b8be-d619-4bab-b431-217903b98055 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:04.672011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff04b63-cfe4-49f3-a070-3a0cd93c0b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:04.672725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff04b63-cfe4-49f3-a070-3a0cd93c0b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.673019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff04b63-cfe4-49f3-a070-3a0cd93c0b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.673294 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff04b63-cfe4-49f3-a070-3a0cd93c0b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.674017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff04b63-cfe4-49f3-a070-3a0cd93c0b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:04.676032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff04b63-cfe4-49f3-a070-3a0cd93c0b77 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:04.737157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1092/2170] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:04 2022] DELETE /identity/v3/users/508fab5e46e64100b5924d03fcf88c8d => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:04.743144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:04.743144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:04.812732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6028ab-1f6f-4abb-b87b-e5d0ac37db22 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:04.817590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff4fad-7f4c-4fc5-afde-a9d53340f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:04.818270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff4fad-7f4c-4fc5-afde-a9d53340f8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.818618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff4fad-7f4c-4fc5-afde-a9d53340f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.818809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff4fad-7f4c-4fc5-afde-a9d53340f8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.819536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff4fad-7f4c-4fc5-afde-a9d53340f8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:04.821482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff4fad-7f4c-4fc5-afde-a9d53340f8f0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:04.885504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1079/2171] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:04 2022] POST /identity/v3/projects => generated 373 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:04.889713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff04b63-cfe4-49f3-a070-3a0cd93c0b77 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:04.978545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff04b63-cfe4-49f3-a070-3a0cd93c0b77 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:04.978545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f6969-1837-42e4-827d-ac13b4228d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:04.978545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f6969-1837-42e4-827d-ac13b4228d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:04.978545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f6969-1837-42e4-827d-ac13b4228d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:04.978991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f6969-1837-42e4-827d-ac13b4228d95 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:04.978991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f6969-1837-42e4-827d-ac13b4228d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:04.979143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f6969-1837-42e4-827d-ac13b4228d95 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:05.032083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1093/2172] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:04 2022] POST /identity/v3/users => generated 305 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:05.046452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff4fad-7f4c-4fc5-afde-a9d53340f8f0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:05.069797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff4fad-7f4c-4fc5-afde-a9d53340f8f0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:05.085866 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae96af-a6fa-4a1c-b47b-2bdfc4cfab29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:05.098183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae96af-a6fa-4a1c-b47b-2bdfc4cfab29 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.099667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae96af-a6fa-4a1c-b47b-2bdfc4cfab29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.100128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae96af-a6fa-4a1c-b47b-2bdfc4cfab29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.101929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae96af-a6fa-4a1c-b47b-2bdfc4cfab29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:05.105509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae96af-a6fa-4a1c-b47b-2bdfc4cfab29 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:05.121695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1094/2173] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:05.127322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae96af-a6fa-4a1c-b47b-2bdfc4cfab29 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:05.150088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae96af-a6fa-4a1c-b47b-2bdfc4cfab29 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:05.163414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4141e-808d-41b3-bde9-8568c2b9ba86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:05.163656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4141e-808d-41b3-bde9-8568c2b9ba86 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.163962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4141e-808d-41b3-bde9-8568c2b9ba86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.164272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4141e-808d-41b3-bde9-8568c2b9ba86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.165144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4141e-808d-41b3-bde9-8568c2b9ba86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:05.172107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4141e-808d-41b3-bde9-8568c2b9ba86 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:05.201731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1095/2174] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:05.202738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4141e-808d-41b3-bde9-8568c2b9ba86 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:05.234194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1080/2175] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:04 2022] POST /identity/v3/users => generated 287 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:06:05.239676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f6969-1837-42e4-827d-ac13b4228d95 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:05.274952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4141e-808d-41b3-bde9-8568c2b9ba86 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:05.286122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edeb6c1-306f-4928-83ea-09757cfed635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:05.287801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edeb6c1-306f-4928-83ea-09757cfed635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.288537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edeb6c1-306f-4928-83ea-09757cfed635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.290662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edeb6c1-306f-4928-83ea-09757cfed635 admin admin] PATH_INFO: `/v3/projects/c039f6f3b515439992f7f7b0a39c5a4c/users/712d3f65099c43c0a97548d4fc452da5/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.292454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f6969-1837-42e4-827d-ac13b4228d95 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:05.295311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987c247-8b12-4686-896a-4d74ff6d2f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:05.296532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987c247-8b12-4686-896a-4d74ff6d2f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.296818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987c247-8b12-4686-896a-4d74ff6d2f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.297099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987c247-8b12-4686-896a-4d74ff6d2f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.298060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987c247-8b12-4686-896a-4d74ff6d2f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:05.299538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987c247-8b12-4686-896a-4d74ff6d2f4b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:05.311721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1081/2176] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:05.317761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987c247-8b12-4686-896a-4d74ff6d2f4b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:05.322500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edeb6c1-306f-4928-83ea-09757cfed635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c039f6f3b515439992f7f7b0a39c5a4c, user_id=712d3f65099c43c0a97548d4fc452da5, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:05.326529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edeb6c1-306f-4928-83ea-09757cfed635 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:05.336327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987c247-8b12-4686-896a-4d74ff6d2f4b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:05.360332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47045129-8820-432c-bc80-4d2d59a45757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:05.361442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47045129-8820-432c-bc80-4d2d59a45757 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.361932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47045129-8820-432c-bc80-4d2d59a45757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.362514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47045129-8820-432c-bc80-4d2d59a45757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.363363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47045129-8820-432c-bc80-4d2d59a45757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:05.367631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47045129-8820-432c-bc80-4d2d59a45757 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:05.381571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1082/2177] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:05.391156 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47045129-8820-432c-bc80-4d2d59a45757 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:05.404350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47045129-8820-432c-bc80-4d2d59a45757 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:05.409172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:05.410440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.410440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.410732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] PATH_INFO: `/v3/projects/c055bf316afa48588fb857b3b5bfc61f/users/d12661d2052b448dabd85debcc095155/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.426864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1096/2178] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:05 2022] PUT /identity/v3/projects/c039f6f3b515439992f7f7b0a39c5a4c/users/712d3f65099c43c0a97548d4fc452da5/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:05.439647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c055bf316afa48588fb857b3b5bfc61f, user_id=d12661d2052b448dabd85debcc095155, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:05.442518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:05.445015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243720-b3ce-4ff8-ae81-add0e5b690ac None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.445382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243720-b3ce-4ff8-ae81-add0e5b690ac None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.445957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243720-b3ce-4ff8-ae81-add0e5b690ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.493341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cd243720-b3ce-4ff8-ae81-add0e5b690ac None None] MFA Rules not processed for user `712d3f65099c43c0a97548d4fc452da5`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:05.496365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cd243720-b3ce-4ff8-ae81-add0e5b690ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:05.589555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1083/2179] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:05 2022] PUT /identity/v3/projects/c055bf316afa48588fb857b3b5bfc61f/users/d12661d2052b448dabd85debcc095155/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:05.590924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:05.593214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:05.606963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1097/2180] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:05.608825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd243720-b3ce-4ff8-ae81-add0e5b690ac None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:05.612329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cd243720-b3ce-4ff8-ae81-add0e5b690ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:05.703609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088bb7f-444c-4525-8224-1e6d0e04dc78 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:05.718462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:05.719460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.719888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.720142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] PATH_INFO: `/v3/projects/1cc6cb68cd664bd888b13f2a9864d868` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.721519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc6cb68cd664bd888b13f2a9864d868)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:05.721802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd243720-b3ce-4ff8-ae81-add0e5b690ac None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:05.723579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:05.726731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:05.727864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:05.728005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:05.728186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] PATH_INFO: `/v3/users/0d157e92270546828c33aaa03247814b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:05.729397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d157e92270546828c33aaa03247814b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:05.731354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:06.051404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc6cb68cd664bd888b13f2a9864d868', 'request_id': 'req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.054441 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d157e92270546828c33aaa03247814b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.055455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d157e92270546828c33aaa03247814b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.056267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d157e92270546828c33aaa03247814b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.058849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc6cb68cd664bd888b13f2a9864d868', 'request_id': 'req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.066865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc6cb68cd664bd888b13f2a9864d868', 'request_id': 'req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.072495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d157e92270546828c33aaa03247814b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.084929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13fdf188-1247-4a9d-95e6-6d15f7cc4c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d157e92270546828c33aaa03247814b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.196737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] The token cache is being invalidate because project 1cc6cb68cd664bd888b13f2a9864d868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:06.197037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.197710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5bdcf9ab-0026-4c3b-9ca0-2a9aafdb3ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:06.199467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1084/2181] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:05 2022] DELETE /identity/v3/projects/1cc6cb68cd664bd888b13f2a9864d868 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:06.202510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcc209-5882-4366-a243-e83a79e8f8f3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:06.202510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcc209-5882-4366-a243-e83a79e8f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:06.202510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcc209-5882-4366-a243-e83a79e8f8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:06.232703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d0dcc209-5882-4366-a243-e83a79e8f8f3 None None] MFA Rules not processed for user `d12661d2052b448dabd85debcc095155`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:06.234438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d0dcc209-5882-4366-a243-e83a79e8f8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:06.251798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:06.307268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1085/2182] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:06.308129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcc209-5882-4366-a243-e83a79e8f8f3 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:06.317055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d0dcc209-5882-4366-a243-e83a79e8f8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:06.320775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1098/2183] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:05 2022] DELETE /identity/v3/users/0d157e92270546828c33aaa03247814b => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:06.320775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c4e3-40ab-4976-9ed1-2490bab44dd7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:06.320775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c4e3-40ab-4976-9ed1-2490bab44dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:06.320775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c4e3-40ab-4976-9ed1-2490bab44dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:06.355900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ec50c4e3-40ab-4976-9ed1-2490bab44dd7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:06.357665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ec50c4e3-40ab-4976-9ed1-2490bab44dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:06.403717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcc209-5882-4366-a243-e83a79e8f8f3 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:06.446523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1099/2184] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:06.448126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50c4e3-40ab-4976-9ed1-2490bab44dd7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:06.451054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ec50c4e3-40ab-4976-9ed1-2490bab44dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:06.494512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952349e0-92b5-48aa-a53e-73f1627892e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:06.495566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952349e0-92b5-48aa-a53e-73f1627892e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:06.496872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952349e0-92b5-48aa-a53e-73f1627892e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:06.497433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952349e0-92b5-48aa-a53e-73f1627892e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:06.498605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952349e0-92b5-48aa-a53e-73f1627892e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:06.502097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952349e0-92b5-48aa-a53e-73f1627892e4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:06.564046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50c4e3-40ab-4976-9ed1-2490bab44dd7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:06.570504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a9bdb-7947-4e97-9b91-1f9b95efcba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:06.571743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a9bdb-7947-4e97-9b91-1f9b95efcba1 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:06.572061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a9bdb-7947-4e97-9b91-1f9b95efcba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:06.574042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a9bdb-7947-4e97-9b91-1f9b95efcba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:06.641869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:06.656630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1086/2185] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:06 2022] POST /identity/v3/projects => generated 409 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:06.659135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e15e9c-3410-48c1-9400-24c70c99e67e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:06.659674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e15e9c-3410-48c1-9400-24c70c99e67e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:06.660149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e15e9c-3410-48c1-9400-24c70c99e67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:06.671708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a9bdb-7947-4e97-9b91-1f9b95efcba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:06.678665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a9bdb-7947-4e97-9b91-1f9b95efcba1 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:06.699854 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-19e15e9c-3410-48c1-9400-24c70c99e67e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:06.702285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-19e15e9c-3410-48c1-9400-24c70c99e67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:06.747965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1100/2186] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:06.747965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee045-0d8f-4bbb-ab20-b028b8b7de92 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:06.747965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee045-0d8f-4bbb-ab20-b028b8b7de92 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:06.747965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee045-0d8f-4bbb-ab20-b028b8b7de92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:06.814170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-389ee045-0d8f-4bbb-ab20-b028b8b7de92 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:06.822893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-389ee045-0d8f-4bbb-ab20-b028b8b7de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:06.882166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1087/2187] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:06.891606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e15e9c-3410-48c1-9400-24c70c99e67e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:06.912527 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e15e9c-3410-48c1-9400-24c70c99e67e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:06.922524 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:06.924137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:06.924590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:06.924986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:06.925804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:06.928588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:06.960456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1101/2188] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:06.962170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ee045-0d8f-4bbb-ab20-b028b8b7de92 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:06.964719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-389ee045-0d8f-4bbb-ab20-b028b8b7de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:06.991297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ee045-0d8f-4bbb-ab20-b028b8b7de92 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:06.996074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:06.997061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:06.997415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:06.997848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] PATH_INFO: `/v3/users/d12661d2052b448dabd85debcc095155` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:06.999194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12661d2052b448dabd85debcc095155)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:07.001674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:07.083494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1088/2189] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:06 2022] POST /identity/v3/users => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:07.084492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:07.086338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:07.193825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879c630-4a92-45c2-93ce-e29244de8e1f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:07.197911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f9e0d-e11c-4004-9fd1-f6ad0b4ef928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:07.198601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f9e0d-e11c-4004-9fd1-f6ad0b4ef928 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:07.198880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f9e0d-e11c-4004-9fd1-f6ad0b4ef928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:07.199161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f9e0d-e11c-4004-9fd1-f6ad0b4ef928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:07.210035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f9e0d-e11c-4004-9fd1-f6ad0b4ef928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:07.211575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f9e0d-e11c-4004-9fd1-f6ad0b4ef928 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:07.224208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1089/2190] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:07.225060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f9e0d-e11c-4004-9fd1-f6ad0b4ef928 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:07.245335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f9e0d-e11c-4004-9fd1-f6ad0b4ef928 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:07.269022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:07.269936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:07.270268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:07.270503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:07.271281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:07.271743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:07.273595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:07.292792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1090/2191] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:07.292792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:07.302644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd767013-6a75-4dc8-a1ed-a854a6b33f59 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:07.309756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181871b7-ad39-4bce-9f87-8e736b05e0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:07.310464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181871b7-ad39-4bce-9f87-8e736b05e0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:07.310747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181871b7-ad39-4bce-9f87-8e736b05e0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:07.311075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181871b7-ad39-4bce-9f87-8e736b05e0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:07.311939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181871b7-ad39-4bce-9f87-8e736b05e0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:07.314313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181871b7-ad39-4bce-9f87-8e736b05e0c3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:07.335411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1091/2192] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:07.335411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181871b7-ad39-4bce-9f87-8e736b05e0c3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:07.347479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181871b7-ad39-4bce-9f87-8e736b05e0c3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:07.363309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:07.363309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:07.363309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:07.363309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:07.363679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:07.365327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:07.378935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12661d2052b448dabd85debcc095155'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:07.380104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12661d2052b448dabd85debcc095155'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:07.380884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12661d2052b448dabd85debcc095155'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:07.395401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12661d2052b448dabd85debcc095155'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:07.402690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12661d2052b448dabd85debcc095155'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:07.496082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1092/2193] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:07 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:07.497887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:07.505599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:07.524219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1102/2194] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:06 2022] DELETE /identity/v3/users/d12661d2052b448dabd85debcc095155 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:07.526629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:07.531439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:07.632579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdbead-12bf-49bf-92be-ce1c972af669 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:07.644844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67369fec-1de0-476b-a3c8-11e20711ddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:07.644844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369fec-1de0-476b-a3c8-11e20711ddee admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:07.644844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369fec-1de0-476b-a3c8-11e20711ddee admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:07.644844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369fec-1de0-476b-a3c8-11e20711ddee admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:07.645642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67369fec-1de0-476b-a3c8-11e20711ddee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:07.645642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67369fec-1de0-476b-a3c8-11e20711ddee admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:07.668410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547aba-8091-412d-af03-efe36f1fb8fd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:07.675865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae753e-a883-4e70-8b2c-d57b921a8091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:07.676159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae753e-a883-4e70-8b2c-d57b921a8091 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:07.677290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae753e-a883-4e70-8b2c-d57b921a8091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:07.677290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae753e-a883-4e70-8b2c-d57b921a8091 admin admin] PATH_INFO: `/v3/projects/e27c12f4053a43198eee47036133e9b4/users/5f43bbacc8594156a568ae90334d665f/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:07.709601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae753e-a883-4e70-8b2c-d57b921a8091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27c12f4053a43198eee47036133e9b4, user_id=5f43bbacc8594156a568ae90334d665f, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:07.710084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae753e-a883-4e70-8b2c-d57b921a8091 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:07.750154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1103/2195] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:07 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:07.750154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cd8b9-5340-46b4-8f58-91a453ae9eb9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:07.750154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cd8b9-5340-46b4-8f58-91a453ae9eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:07.750877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cd8b9-5340-46b4-8f58-91a453ae9eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:07.784855 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a15cd8b9-5340-46b4-8f58-91a453ae9eb9 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:07.789942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a15cd8b9-5340-46b4-8f58-91a453ae9eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:07.806735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1104/2196] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:07.808510 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cd8b9-5340-46b4-8f58-91a453ae9eb9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:07.813898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1093/2197] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:07 2022] PUT /identity/v3/projects/e27c12f4053a43198eee47036133e9b4/users/5f43bbacc8594156a568ae90334d665f/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:07.818615 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae753e-a883-4e70-8b2c-d57b921a8091 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:07.840775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cd8b9-5340-46b4-8f58-91a453ae9eb9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:07.989090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae753e-a883-4e70-8b2c-d57b921a8091 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:07.996474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:07.997379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:07.998478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:07.998622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:07.998876 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:07.999567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:07.999889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.000392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.000905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.001207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.002452 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:08.006363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.016268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1105/2198] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:08.016268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:08.025005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:08.050719 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1094/2199] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:08.052700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:08.057628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:08.083029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10d29-696f-4aff-9464-987e39906fca admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:08.088464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:08.090002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.090468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.090854 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] PATH_INFO: `/v3/projects/822ca23f26ad45eb8357d061afb9a03a` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.092885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=822ca23f26ad45eb8357d061afb9a03a)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:08.094903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.138461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28f273-d61b-4b4d-8f4c-2208ce417bcf admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:08.146494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f874f8e-8e2f-40d7-ae1f-a4a2c557dbbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:08.147290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874f8e-8e2f-40d7-ae1f-a4a2c557dbbb service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.147629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874f8e-8e2f-40d7-ae1f-a4a2c557dbbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.148075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874f8e-8e2f-40d7-ae1f-a4a2c557dbbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.161502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874f8e-8e2f-40d7-ae1f-a4a2c557dbbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:08.162989 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874f8e-8e2f-40d7-ae1f-a4a2c557dbbb service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.173912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1106/2200] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:08.174782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f874f8e-8e2f-40d7-ae1f-a4a2c557dbbb service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:08.187006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f874f8e-8e2f-40d7-ae1f-a4a2c557dbbb service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:08.193232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29140202-077d-4919-a40a-df2f26277a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:08.194334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140202-077d-4919-a40a-df2f26277a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.194769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140202-077d-4919-a40a-df2f26277a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.195044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140202-077d-4919-a40a-df2f26277a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.195852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29140202-077d-4919-a40a-df2f26277a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:08.197506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29140202-077d-4919-a40a-df2f26277a91 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.214671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1107/2201] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:08.214671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29140202-077d-4919-a40a-df2f26277a91 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:08.224721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29140202-077d-4919-a40a-df2f26277a91 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:08.250048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822ca23f26ad45eb8357d061afb9a03a', 'request_id': 'req-88cdd11a-b869-4f23-b2bd-0b3f94c73274'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:08.258616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822ca23f26ad45eb8357d061afb9a03a', 'request_id': 'req-88cdd11a-b869-4f23-b2bd-0b3f94c73274'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:08.270580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822ca23f26ad45eb8357d061afb9a03a', 'request_id': 'req-88cdd11a-b869-4f23-b2bd-0b3f94c73274'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:08.317931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:08.353532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] The token cache is being invalidate because project 822ca23f26ad45eb8357d061afb9a03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:08.359387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:08.359591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b7b77-e816-4e44-af10-e4d3f9091fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:08.359759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:08.359899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b7b77-e816-4e44-af10-e4d3f9091fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.359899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b7b77-e816-4e44-af10-e4d3f9091fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.359899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b7b77-e816-4e44-af10-e4d3f9091fc2 admin admin] PATH_INFO: `/v3/projects/e27c12f4053a43198eee47036133e9b4/users/5f43bbacc8594156a568ae90334d665f/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.362514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1095/2202] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:08 2022] DELETE /identity/v3/projects/822ca23f26ad45eb8357d061afb9a03a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:08.363904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b7b77-e816-4e44-af10-e4d3f9091fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27c12f4053a43198eee47036133e9b4, user_id=5f43bbacc8594156a568ae90334d665f, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:08.366526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:08.369642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b7b77-e816-4e44-af10-e4d3f9091fc2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.371164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:08.403279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1108/2203] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:08 2022] PUT /identity/v3/projects/e27c12f4053a43198eee47036133e9b4/users/5f43bbacc8594156a568ae90334d665f/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:08.403279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e9408-b59d-4f8c-a397-fb6d4d8aa55c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.403279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e9408-b59d-4f8c-a397-fb6d4d8aa55c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.403279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e9408-b59d-4f8c-a397-fb6d4d8aa55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.438883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdd11a-b869-4f23-b2bd-0b3f94c73274 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:08.461201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5a7e9408-b59d-4f8c-a397-fb6d4d8aa55c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:08.464501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5a7e9408-b59d-4f8c-a397-fb6d4d8aa55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:08.569701 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1109/2204] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:08.571735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597b63b-a2b4-4897-8b4a-dca95976370d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.572190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597b63b-a2b4-4897-8b4a-dca95976370d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.572595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597b63b-a2b4-4897-8b4a-dca95976370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.581504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf84e3e-4f29-455e-8fdb-80ada09f97a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:08.583392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf84e3e-4f29-455e-8fdb-80ada09f97a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.584086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf84e3e-4f29-455e-8fdb-80ada09f97a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.584737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf84e3e-4f29-455e-8fdb-80ada09f97a1 admin admin] PATH_INFO: `/v3/projects/d9797e61900c49afb98864c08dd7502e/users/a5e6a2917c5946aeb01e9619e7254c49/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.605721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf84e3e-4f29-455e-8fdb-80ada09f97a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9797e61900c49afb98864c08dd7502e, user_id=a5e6a2917c5946aeb01e9619e7254c49, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:08.609776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf84e3e-4f29-455e-8fdb-80ada09f97a1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.622092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b597b63b-a2b4-4897-8b4a-dca95976370d None None] MFA Rules not processed for user `5f43bbacc8594156a568ae90334d665f`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:08.623567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b597b63b-a2b4-4897-8b4a-dca95976370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:08.708796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1110/2205] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:08 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:08.708796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1341b-bba7-4e26-8775-a9ca2b0915ed None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.708796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1341b-bba7-4e26-8775-a9ca2b0915ed None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.708796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1341b-bba7-4e26-8775-a9ca2b0915ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.726185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c0d1341b-bba7-4e26-8775-a9ca2b0915ed None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:08.727614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c0d1341b-bba7-4e26-8775-a9ca2b0915ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:08.736079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1111/2206] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:08.736947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1341b-bba7-4e26-8775-a9ca2b0915ed None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:08.738287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c0d1341b-bba7-4e26-8775-a9ca2b0915ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:08.818164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1341b-bba7-4e26-8775-a9ca2b0915ed None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:08.822089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75118c3f-c9aa-42a8-aa88-1eff4789ba3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:08.822859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118c3f-c9aa-42a8-aa88-1eff4789ba3c service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.823187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118c3f-c9aa-42a8-aa88-1eff4789ba3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.823504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118c3f-c9aa-42a8-aa88-1eff4789ba3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.834393 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75118c3f-c9aa-42a8-aa88-1eff4789ba3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:08.835904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75118c3f-c9aa-42a8-aa88-1eff4789ba3c service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.847504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1112/2207] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:08 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:08.848373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75118c3f-c9aa-42a8-aa88-1eff4789ba3c service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:08.860664 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75118c3f-c9aa-42a8-aa88-1eff4789ba3c service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:08.890761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:08.892671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.893341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.893978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.894887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1096/2208] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:08 2022] PUT /identity/v3/projects/d9797e61900c49afb98864c08dd7502e/users/a5e6a2917c5946aeb01e9619e7254c49/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:08.895860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:08.896536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:08.899995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:08.906874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef698c2-25fd-4aa3-809b-f5176b5cb225 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:08.907437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef698c2-25fd-4aa3-809b-f5176b5cb225 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:08.907945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef698c2-25fd-4aa3-809b-f5176b5cb225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:08.935597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1113/2209] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:08.941450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:08.954043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6ef698c2-25fd-4aa3-809b-f5176b5cb225 None None] MFA Rules not processed for user `a5e6a2917c5946aeb01e9619e7254c49`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:08.956163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6ef698c2-25fd-4aa3-809b-f5176b5cb225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:08.958801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9a4d-2aa2-4681-92bf-91444beffa93 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:09.043278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e422-6c13-43b5-b82a-e5d404bc09a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:09.043278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e422-6c13-43b5-b82a-e5d404bc09a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:09.043278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e422-6c13-43b5-b82a-e5d404bc09a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:09.043278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e422-6c13-43b5-b82a-e5d404bc09a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:09.044317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e422-6c13-43b5-b82a-e5d404bc09a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:09.044317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e422-6c13-43b5-b82a-e5d404bc09a3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:09.070187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:09.084598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1097/2210] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:09.085562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef698c2-25fd-4aa3-809b-f5176b5cb225 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:09.109477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef698c2-25fd-4aa3-809b-f5176b5cb225 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:09.125819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:09.126588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:09.127026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:09.127214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:09.129902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:09.133432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:09.234501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1114/2211] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:08 2022] POST /identity/v3/projects => generated 385 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:09.234501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e422-6c13-43b5-b82a-e5d404bc09a3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:09.262836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1098/2212] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:09 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:09.264889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:09.271455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:09.354358 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e422-6c13-43b5-b82a-e5d404bc09a3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:09.412143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed03154-6ba8-43f4-b5ac-7df199f6226f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:09.422855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:09.424690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:09.425344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:09.425978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] PATH_INFO: `/v3/projects/c055bf316afa48588fb857b3b5bfc61f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:09.428622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c055bf316afa48588fb857b3b5bfc61f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:09.438042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:09.513474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2735-560d-49f0-8966-f0416e11e06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:09.514861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2735-560d-49f0-8966-f0416e11e06d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:09.515198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2735-560d-49f0-8966-f0416e11e06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:09.515576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2735-560d-49f0-8966-f0416e11e06d admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:09.516356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f2735-560d-49f0-8966-f0416e11e06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:09.518486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f2735-560d-49f0-8966-f0416e11e06d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:09.641618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1115/2213] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:09 2022] POST /identity/v3/users => generated 293 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:09.643243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2735-560d-49f0-8966-f0416e11e06d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:09.663203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2735-560d-49f0-8966-f0416e11e06d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:09.678639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c055bf316afa48588fb857b3b5bfc61f', 'request_id': 'req-a6297ad4-3cde-409f-927c-713418d95768'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:09.690720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c055bf316afa48588fb857b3b5bfc61f', 'request_id': 'req-a6297ad4-3cde-409f-927c-713418d95768'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:09.709549 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c055bf316afa48588fb857b3b5bfc61f', 'request_id': 'req-a6297ad4-3cde-409f-927c-713418d95768'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:09.788315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12f97-3926-4468-a8f3-897af7ce3a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:09.789491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12f97-3926-4468-a8f3-897af7ce3a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:09.789983 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12f97-3926-4468-a8f3-897af7ce3a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:09.790712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12f97-3926-4468-a8f3-897af7ce3a23 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:09.792911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa12f97-3926-4468-a8f3-897af7ce3a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:09.797951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa12f97-3926-4468-a8f3-897af7ce3a23 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:09.842436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] The token cache is being invalidate because project c055bf316afa48588fb857b3b5bfc61f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:09.843353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:09.846791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:09.850458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1099/2214] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:09 2022] DELETE /identity/v3/projects/c055bf316afa48588fb857b3b5bfc61f => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:09.852130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:09.855298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:09.886005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6297ad4-3cde-409f-927c-713418d95768 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:09.901724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:09.904070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:09.905029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:09.906501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:09.911398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:09.921144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:09.955757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1100/2215] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:09.956857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:09.960610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:10.017789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1116/2216] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:09 2022] POST /identity/v3/users => generated 293 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:10.018801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12f97-3926-4468-a8f3-897af7ce3a23 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:10.044498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12f97-3926-4468-a8f3-897af7ce3a23 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:10.050558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:10.058273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.059036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.059133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.071601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:10.073359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:10.087014 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1117/2217] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:10.088069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:10.090717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:10.106458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf399c-8792-4b90-909c-7374c6011ab2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:10.125639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6f781-20ae-4189-910f-a843ef35467d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:10.139421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f781-20ae-4189-910f-a843ef35467d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.139421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f781-20ae-4189-910f-a843ef35467d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.139421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f781-20ae-4189-910f-a843ef35467d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.139421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6f781-20ae-4189-910f-a843ef35467d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:10.139421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6f781-20ae-4189-910f-a843ef35467d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:10.140528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627def3-a3bc-4a6f-84b3-22339b8a59cd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:10.144543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:10.144787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.145074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.145348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.150494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1118/2218] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:10.150727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:10.150832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6f781-20ae-4189-910f-a843ef35467d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:10.162335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6f781-20ae-4189-910f-a843ef35467d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:10.169993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:10.171714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.173719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.173719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] PATH_INFO: `/v3/projects/a1a9301a62af442694ed54c69d58928f/users/85e071aa9a6f453f96d1923327ae306f/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.198269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a9301a62af442694ed54c69d58928f, user_id=85e071aa9a6f453f96d1923327ae306f, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:10.205631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:10.246879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:10.249084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:10.259432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1101/2219] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:09 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:10.260567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:10.276152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5641f-299e-4078-963d-642d625bdb73 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:10.295812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5fbc8-62f3-4c4e-9ae2-63025c3de4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:10.295812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5fbc8-62f3-4c4e-9ae2-63025c3de4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.295812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5fbc8-62f3-4c4e-9ae2-63025c3de4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.295812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5fbc8-62f3-4c4e-9ae2-63025c3de4fd admin admin] PATH_INFO: `/v3/projects/46ba2c9f14ad42358a5b3aa451a709d3/users/9373bd1c34044417887ec1bf1c2ba5c9/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.301035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5fbc8-62f3-4c4e-9ae2-63025c3de4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ba2c9f14ad42358a5b3aa451a709d3, user_id=9373bd1c34044417887ec1bf1c2ba5c9, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:10.304035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5fbc8-62f3-4c4e-9ae2-63025c3de4fd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:10.322650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1119/2220] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:10 2022] PUT /identity/v3/projects/a1a9301a62af442694ed54c69d58928f/users/85e071aa9a6f453f96d1923327ae306f/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:10.332566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:10.336923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:10.444094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1102/2221] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:10 2022] PUT /identity/v3/projects/46ba2c9f14ad42358a5b3aa451a709d3/users/9373bd1c34044417887ec1bf1c2ba5c9/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:10.445701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb46878-8674-442e-bd51-051b938d59e1 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.445992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb46878-8674-442e-bd51-051b938d59e1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.446268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb46878-8674-442e-bd51-051b938d59e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.530742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8bb46878-8674-442e-bd51-051b938d59e1 None None] MFA Rules not processed for user `85e071aa9a6f453f96d1923327ae306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:10.532056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8bb46878-8674-442e-bd51-051b938d59e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:10.589417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e106f-69db-4fb4-bae6-0f91f9621b4d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:10.726932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1103/2222] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:10.726932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb46878-8674-442e-bd51-051b938d59e1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:10.749138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:10.750650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.751643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.752993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] PATH_INFO: `/v3/users/f08669c1c4e94d4187d15736462a4215` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.756466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08669c1c4e94d4187d15736462a4215)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:10.761386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:10.870355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb46878-8674-442e-bd51-051b938d59e1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:10.878463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943ee1-a3c2-4118-8f44-b9a5ea3a2f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:10.879693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943ee1-a3c2-4118-8f44-b9a5ea3a2f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.880190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943ee1-a3c2-4118-8f44-b9a5ea3a2f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.880682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943ee1-a3c2-4118-8f44-b9a5ea3a2f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.882810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87943ee1-a3c2-4118-8f44-b9a5ea3a2f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:10.885338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87943ee1-a3c2-4118-8f44-b9a5ea3a2f7c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:10.907400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1104/2223] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:10.907400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943ee1-a3c2-4118-8f44-b9a5ea3a2f7c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:10.928002 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943ee1-a3c2-4118-8f44-b9a5ea3a2f7c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:10.937863 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a593a0-4036-4ed5-915b-8b70e7400e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:10.939009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a593a0-4036-4ed5-915b-8b70e7400e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:10.939780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a593a0-4036-4ed5-915b-8b70e7400e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:10.940428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a593a0-4036-4ed5-915b-8b70e7400e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:10.942290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a593a0-4036-4ed5-915b-8b70e7400e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:10.945987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a593a0-4036-4ed5-915b-8b70e7400e7b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:11.121052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1105/2224] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:10 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:11.121052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a593a0-4036-4ed5-915b-8b70e7400e7b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:11.168312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08669c1c4e94d4187d15736462a4215'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:11.169512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08669c1c4e94d4187d15736462a4215'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:11.170401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08669c1c4e94d4187d15736462a4215'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:11.198860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08669c1c4e94d4187d15736462a4215'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:11.213450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-99544ed0-90b7-451d-a8ce-b54796844824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08669c1c4e94d4187d15736462a4215'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:11.227752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a593a0-4036-4ed5-915b-8b70e7400e7b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:11.236620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:11.238174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.238681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.240438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] PATH_INFO: `/v3/projects/46ba2c9f14ad42358a5b3aa451a709d3/users/9373bd1c34044417887ec1bf1c2ba5c9/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.243585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ba2c9f14ad42358a5b3aa451a709d3, user_id=9373bd1c34044417887ec1bf1c2ba5c9, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:11.249800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:11.272915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1106/2225] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:11 2022] PUT /identity/v3/projects/46ba2c9f14ad42358a5b3aa451a709d3/users/9373bd1c34044417887ec1bf1c2ba5c9/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:11.274397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:11.281148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:11.368414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1120/2226] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:10 2022] DELETE /identity/v3/users/f08669c1c4e94d4187d15736462a4215 => generated 0 bytes in 1039 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:11.370124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487da30-e406-4e36-afa9-831e6bcacfef None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.371432 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487da30-e406-4e36-afa9-831e6bcacfef None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.371883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487da30-e406-4e36-afa9-831e6bcacfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.409438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2213d-919f-4039-b994-09f8e399a752 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:11.455977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f487da30-e406-4e36-afa9-831e6bcacfef None None] MFA Rules not processed for user `9373bd1c34044417887ec1bf1c2ba5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:11.457628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f487da30-e406-4e36-afa9-831e6bcacfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:11.523061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:11.566739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:11.567519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.567756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.568035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.569063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:11.571036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:11.590846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1121/2227] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:11.594613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2237-a752-42c5-8fda-2c57d0b3b4a9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.594613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2237-a752-42c5-8fda-2c57d0b3b4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.594613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2237-a752-42c5-8fda-2c57d0b3b4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.627218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-892e2237-a752-42c5-8fda-2c57d0b3b4a9 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:11.628794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-892e2237-a752-42c5-8fda-2c57d0b3b4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:11.637865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1122/2228] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:11.639364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e2237-a752-42c5-8fda-2c57d0b3b4a9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:11.640848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-892e2237-a752-42c5-8fda-2c57d0b3b4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:11.713907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1107/2229] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:11 2022] POST /identity/v3/users => generated 293 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:11.715536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:11.719553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:11.729095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e2237-a752-42c5-8fda-2c57d0b3b4a9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:11.733803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc441b3e-b3fc-458c-a0cd-a50f475747d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:11.740898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc441b3e-b3fc-458c-a0cd-a50f475747d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.740898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc441b3e-b3fc-458c-a0cd-a50f475747d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.740898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc441b3e-b3fc-458c-a0cd-a50f475747d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.746151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc441b3e-b3fc-458c-a0cd-a50f475747d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:11.749520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc441b3e-b3fc-458c-a0cd-a50f475747d8 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:11.771178 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1123/2230] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:11.773041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc441b3e-b3fc-458c-a0cd-a50f475747d8 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:11.793554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc441b3e-b3fc-458c-a0cd-a50f475747d8 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:11.802056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2804ce-c75d-4f64-96d9-681ada74440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:11.815422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2804ce-c75d-4f64-96d9-681ada74440e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.815422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2804ce-c75d-4f64-96d9-681ada74440e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.815422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2804ce-c75d-4f64-96d9-681ada74440e admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.815422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2804ce-c75d-4f64-96d9-681ada74440e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:11.816047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9b4c0-eb43-436a-ab20-1abf44891afb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:11.816151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2804ce-c75d-4f64-96d9-681ada74440e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:11.816243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8b705-1748-4ef2-9b39-6bce077095e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:11.816243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8b705-1748-4ef2-9b39-6bce077095e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.816243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8b705-1748-4ef2-9b39-6bce077095e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.816243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8b705-1748-4ef2-9b39-6bce077095e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.827854 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8b705-1748-4ef2-9b39-6bce077095e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:11.828898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1124/2231] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:11.830076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8b705-1748-4ef2-9b39-6bce077095e7 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:11.833587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2804ce-c75d-4f64-96d9-681ada74440e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:11.844753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1108/2232] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:11.851723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2804ce-c75d-4f64-96d9-681ada74440e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:11.859104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829bb66-28ed-4a42-bf5b-ff5bac18de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:11.860775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829bb66-28ed-4a42-bf5b-ff5bac18de56 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.861282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829bb66-28ed-4a42-bf5b-ff5bac18de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.861903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829bb66-28ed-4a42-bf5b-ff5bac18de56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.863369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829bb66-28ed-4a42-bf5b-ff5bac18de56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:11.866268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829bb66-28ed-4a42-bf5b-ff5bac18de56 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:11.890612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1125/2233] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:11.892172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8b705-1748-4ef2-9b39-6bce077095e7 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:11.906159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8b705-1748-4ef2-9b39-6bce077095e7 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:11.912618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e253a-526b-46ef-8822-f671a58b257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:11.913806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e253a-526b-46ef-8822-f671a58b257b admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:11.914266 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e253a-526b-46ef-8822-f671a58b257b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:11.914634 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e253a-526b-46ef-8822-f671a58b257b admin admin] PATH_INFO: `/v3/projects/00194ee4cc0948e0b603e63373126207/users/e47bcf3cd7ff41399433cb2c0a186ade/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:11.947966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e253a-526b-46ef-8822-f671a58b257b admin admin] RBAC: Authorizing `identity:create_grant(project_id=00194ee4cc0948e0b603e63373126207, user_id=e47bcf3cd7ff41399433cb2c0a186ade, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:11.953107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e253a-526b-46ef-8822-f671a58b257b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:12.019557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:12.058727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1109/2234] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:11 2022] PUT /identity/v3/projects/00194ee4cc0948e0b603e63373126207/users/e47bcf3cd7ff41399433cb2c0a186ade/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:12.072733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ee92-2df7-419f-98b7-907becd8dccc None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:12.073621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ee92-2df7-419f-98b7-907becd8dccc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:12.074080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ee92-2df7-419f-98b7-907becd8dccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:12.082528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e253a-526b-46ef-8822-f671a58b257b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:12.141203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-89a3ee92-2df7-419f-98b7-907becd8dccc None None] MFA Rules not processed for user `e47bcf3cd7ff41399433cb2c0a186ade`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:12.144259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-89a3ee92-2df7-419f-98b7-907becd8dccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:12.174599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e253a-526b-46ef-8822-f671a58b257b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:12.187895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:12.189178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:12.189644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:12.190874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:12.194009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:12.275864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:12.275864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:12.280880 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1126/2235] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:12.286182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ee92-2df7-419f-98b7-907becd8dccc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:12.286588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1110/2236] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:06:12.373919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ee92-2df7-419f-98b7-907becd8dccc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:12.378895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414877-043f-4691-b643-6cd73d02b0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:12.379610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414877-043f-4691-b643-6cd73d02b0ac admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:12.379895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414877-043f-4691-b643-6cd73d02b0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:12.380169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414877-043f-4691-b643-6cd73d02b0ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:12.380870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed414877-043f-4691-b643-6cd73d02b0ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:12.382825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed414877-043f-4691-b643-6cd73d02b0ac admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:12.421984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:12.424381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:12.451502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfdd91-e5ed-47b3-8314-b263fbced1cd service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:12.458377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:12.459863 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:12.460105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:12.460532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] PATH_INFO: `/v3/projects/3a1f4a376e804f409f452198d6e186c9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:12.461684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1f4a376e804f409f452198d6e186c9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:12.463784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:12.517002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1127/2237] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:12 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:06:12.520078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414877-043f-4691-b643-6cd73d02b0ac admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:12.631838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1f4a376e804f409f452198d6e186c9', 'request_id': 'req-79fde3c3-f495-4235-ab48-da1f62a51112'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:12.633342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414877-043f-4691-b643-6cd73d02b0ac admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:12.638613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1f4a376e804f409f452198d6e186c9', 'request_id': 'req-79fde3c3-f495-4235-ab48-da1f62a51112'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:12.647107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1f4a376e804f409f452198d6e186c9', 'request_id': 'req-79fde3c3-f495-4235-ab48-da1f62a51112'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:12.713650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18619081-861a-4353-aae7-581609227210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:12.714723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18619081-861a-4353-aae7-581609227210 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:12.715160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18619081-861a-4353-aae7-581609227210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:12.715612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18619081-861a-4353-aae7-581609227210 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:12.716484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18619081-861a-4353-aae7-581609227210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:12.719242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18619081-861a-4353-aae7-581609227210 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:12.743333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] The token cache is being invalidate because project 3a1f4a376e804f409f452198d6e186c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:12.744036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:12.745640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:12.749169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1111/2238] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:12 2022] DELETE /identity/v3/projects/3a1f4a376e804f409f452198d6e186c9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:12.754926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:12.756969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:12.772560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fde3c3-f495-4235-ab48-da1f62a51112 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:12.779311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:12.780884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:12.781458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:12.781857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] PATH_INFO: `/v3/users/224b348268894ac2807dc637a7a39574` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:12.783956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=224b348268894ac2807dc637a7a39574)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:12.786179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:12.829154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1128/2239] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:12 2022] POST /identity/v3/users => generated 299 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:12.833368 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18619081-861a-4353-aae7-581609227210 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:12.835182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-18619081-861a-4353-aae7-581609227210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:12.848401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18619081-861a-4353-aae7-581609227210 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:12.855176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb958-7da1-4eb0-815e-9022aecec12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:12.856105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb958-7da1-4eb0-815e-9022aecec12c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:12.856493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb958-7da1-4eb0-815e-9022aecec12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:12.856897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb958-7da1-4eb0-815e-9022aecec12c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:12.857927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bb958-7da1-4eb0-815e-9022aecec12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:12.859842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bb958-7da1-4eb0-815e-9022aecec12c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:12.872720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1129/2240] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:12.877737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb958-7da1-4eb0-815e-9022aecec12c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:12.891628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb958-7da1-4eb0-815e-9022aecec12c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:12.900123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:12.901737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:12.902246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:12.902752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] PATH_INFO: `/v3/projects/343648df590d40a1b8308f397a446dcf/users/3fbf201a63ad4598989dd0309a67bba5/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:12.933465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=343648df590d40a1b8308f397a446dcf, user_id=3fbf201a63ad4598989dd0309a67bba5, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:12.937795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:13.093956 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224b348268894ac2807dc637a7a39574'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:13.094739 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224b348268894ac2807dc637a7a39574'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:13.121904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1130/2241] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:12 2022] PUT /identity/v3/projects/343648df590d40a1b8308f397a446dcf/users/3fbf201a63ad4598989dd0309a67bba5/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:13.127670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:13.130666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:13.167636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224b348268894ac2807dc637a7a39574'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:13.179864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224b348268894ac2807dc637a7a39574'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:13.180964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224b348268894ac2807dc637a7a39574'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:13.210999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1112/2242] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:12 2022] DELETE /identity/v3/users/224b348268894ac2807dc637a7a39574 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:13.259859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac47de3-84ed-4b8b-a093-048a963dffd0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:13.360903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc0bfa-014e-4490-8e2b-08ae2d6d2cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:13.362104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc0bfa-014e-4490-8e2b-08ae2d6d2cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:13.362482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc0bfa-014e-4490-8e2b-08ae2d6d2cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:13.362854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc0bfa-014e-4490-8e2b-08ae2d6d2cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:13.364268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc0bfa-014e-4490-8e2b-08ae2d6d2cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:13.366325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc0bfa-014e-4490-8e2b-08ae2d6d2cbc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:13.384364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1131/2243] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:13.385625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:13.397241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa567e-6ee6-4c77-aef6-8895e5d0c5b3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:13.405018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:13.406034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:13.406355 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:13.406628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] PATH_INFO: `/v3/projects/343648df590d40a1b8308f397a446dcf/users/3fbf201a63ad4598989dd0309a67bba5/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:13.413078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=343648df590d40a1b8308f397a446dcf, user_id=3fbf201a63ad4598989dd0309a67bba5, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:13.416856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:13.437205 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1113/2244] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:13 2022] PUT /identity/v3/projects/343648df590d40a1b8308f397a446dcf/users/3fbf201a63ad4598989dd0309a67bba5/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:13.448257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f9f01-9988-4090-b1e8-5ee7c4c3b3b5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:13.449527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f9f01-9988-4090-b1e8-5ee7c4c3b3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:13.449527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f9f01-9988-4090-b1e8-5ee7c4c3b3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:13.479910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5f7f9f01-9988-4090-b1e8-5ee7c4c3b3b5 None None] MFA Rules not processed for user `3fbf201a63ad4598989dd0309a67bba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:13.481176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5f7f9f01-9988-4090-b1e8-5ee7c4c3b3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:13.569444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1132/2245] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:13.586286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:13.588727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:13.681855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9edc1c-b3e1-4e4a-ae5d-4e374f8a8609 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:13.688313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7216-f350-43d6-ae0b-3cdc3138e04c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:13.689529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7216-f350-43d6-ae0b-3cdc3138e04c service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:13.689831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7216-f350-43d6-ae0b-3cdc3138e04c service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:13.691945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7216-f350-43d6-ae0b-3cdc3138e04c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:13.693752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d4cb7216-f350-43d6-ae0b-3cdc3138e04c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:13.780175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:13.781626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb7216-f350-43d6-ae0b-3cdc3138e04c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:13.783807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb7216-f350-43d6-ae0b-3cdc3138e04c service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:13.797742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1114/2246] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:13.889100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f9f01-9988-4090-b1e8-5ee7c4c3b3b5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:13.894571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5f7f9f01-9988-4090-b1e8-5ee7c4c3b3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:14.053068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f9f01-9988-4090-b1e8-5ee7c4c3b3b5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:14.058235 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:14.059126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:14.059463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:14.059736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] PATH_INFO: `/v3/projects/92e6a6b0946b410d87127403ac8cfb32` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:14.061593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e6a6b0946b410d87127403ac8cfb32)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:14.063947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:14.205002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e6a6b0946b410d87127403ac8cfb32', 'request_id': 'req-1e593ef6-3059-4019-b47f-bb9bf9496772'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:14.210874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e6a6b0946b410d87127403ac8cfb32', 'request_id': 'req-1e593ef6-3059-4019-b47f-bb9bf9496772'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:14.359788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e6a6b0946b410d87127403ac8cfb32', 'request_id': 'req-1e593ef6-3059-4019-b47f-bb9bf9496772'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:14.372539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] The token cache is being invalidate because project 92e6a6b0946b410d87127403ac8cfb32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:14.373313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:14.373616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e593ef6-3059-4019-b47f-bb9bf9496772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:14.376704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1133/2247] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:13 2022] DELETE /identity/v3/projects/92e6a6b0946b410d87127403ac8cfb32 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:14.381852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88539f4-8bd3-461b-b1c5-f4af287cf2e8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:14.382140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88539f4-8bd3-461b-b1c5-f4af287cf2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:14.382448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88539f4-8bd3-461b-b1c5-f4af287cf2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:14.442608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c88539f4-8bd3-461b-b1c5-f4af287cf2e8 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:14.444221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c88539f4-8bd3-461b-b1c5-f4af287cf2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:14.480692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:14.559896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1115/2248] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:14.572773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f873-7eb4-4173-96bf-d297280c3917 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:14.573230 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f873-7eb4-4173-96bf-d297280c3917 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:14.573691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f873-7eb4-4173-96bf-d297280c3917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:14.612837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5cb8f873-7eb4-4173-96bf-d297280c3917 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:14.615032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5cb8f873-7eb4-4173-96bf-d297280c3917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:14.630200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1134/2249] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:14.636986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88539f4-8bd3-461b-b1c5-f4af287cf2e8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:14.657040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88539f4-8bd3-461b-b1c5-f4af287cf2e8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:14.666120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:14.666819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:14.667107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:14.667422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:14.668282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:14.668632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:14.670550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:14.693341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1116/2250] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 04:06:14.701005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8f873-7eb4-4173-96bf-d297280c3917 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:14.735448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8f873-7eb4-4173-96bf-d297280c3917 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:14.753648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dd2cb-3ec4-492c-a7ee-f3fdcc0c4dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:14.759155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dd2cb-3ec4-492c-a7ee-f3fdcc0c4dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:14.759155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dd2cb-3ec4-492c-a7ee-f3fdcc0c4dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:14.759155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dd2cb-3ec4-492c-a7ee-f3fdcc0c4dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:14.759155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dd2cb-3ec4-492c-a7ee-f3fdcc0c4dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:14.763377 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dd2cb-3ec4-492c-a7ee-f3fdcc0c4dc0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:14.918571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1135/2251] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:14 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:14.920867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:14.934413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2245e6-fa94-46a3-b27a-941c19f1f628 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:15.028716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a6c4-67b5-445c-bab7-cf44e9d69119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:15.029614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a6c4-67b5-445c-bab7-cf44e9d69119 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:15.029902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a6c4-67b5-445c-bab7-cf44e9d69119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:15.030182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a6c4-67b5-445c-bab7-cf44e9d69119 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:15.030973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a6c4-67b5-445c-bab7-cf44e9d69119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:15.034149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a6c4-67b5-445c-bab7-cf44e9d69119 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:15.195450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1117/2252] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:14 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:15.199656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dd2cb-3ec4-492c-a7ee-f3fdcc0c4dc0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:15.211662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dd2cb-3ec4-492c-a7ee-f3fdcc0c4dc0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:15.216643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6069-12e1-4f81-92d7-0d221f66cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:15.217365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6069-12e1-4f81-92d7-0d221f66cff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:15.217648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6069-12e1-4f81-92d7-0d221f66cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:15.217931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6069-12e1-4f81-92d7-0d221f66cff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:15.218680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d6069-12e1-4f81-92d7-0d221f66cff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:15.220360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d6069-12e1-4f81-92d7-0d221f66cff2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:15.232156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1136/2253] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:15.237685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a6c4-67b5-445c-bab7-cf44e9d69119 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:15.257320 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a6c4-67b5-445c-bab7-cf44e9d69119 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:15.265685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f8e51-a4ed-429b-a2a9-c949581f8f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:15.266809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f8e51-a4ed-429b-a2a9-c949581f8f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:15.267347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f8e51-a4ed-429b-a2a9-c949581f8f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:15.267843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f8e51-a4ed-429b-a2a9-c949581f8f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:15.269190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f8e51-a4ed-429b-a2a9-c949581f8f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:15.272149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f8e51-a4ed-429b-a2a9-c949581f8f57 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:15.291807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1118/2254] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:15.296317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6069-12e1-4f81-92d7-0d221f66cff2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:15.307242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6069-12e1-4f81-92d7-0d221f66cff2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:15.312527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:15.315892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:15.315892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:15.315892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] PATH_INFO: `/v3/projects/63b96ecb614d47dd8645a940658c11b1/users/b1aca714eee24db9aaa274bb4d59a064/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:15.331117 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63b96ecb614d47dd8645a940658c11b1, user_id=b1aca714eee24db9aaa274bb4d59a064, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:15.334033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:15.397717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1137/2255] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:15 2022] PUT /identity/v3/projects/63b96ecb614d47dd8645a940658c11b1/users/b1aca714eee24db9aaa274bb4d59a064/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:15.413952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d47cf7-7dc7-481d-b074-3479f58a34d2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:15.414213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d47cf7-7dc7-481d-b074-3479f58a34d2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:15.414496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d47cf7-7dc7-481d-b074-3479f58a34d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:15.446828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b0d47cf7-7dc7-481d-b074-3479f58a34d2 None None] MFA Rules not processed for user `b1aca714eee24db9aaa274bb4d59a064`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:15.448144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b0d47cf7-7dc7-481d-b074-3479f58a34d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:15.526109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1119/2256] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:15.539567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:15.541743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:15.636493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dad7b-f0f3-481e-a099-6359febeca83 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:15.642802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3a326-7bf9-4c3b-8c68-020fde4238cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:15.644043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3a326-7bf9-4c3b-8c68-020fde4238cb service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:15.644551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3a326-7bf9-4c3b-8c68-020fde4238cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:15.645055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3a326-7bf9-4c3b-8c68-020fde4238cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:15.660037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3a326-7bf9-4c3b-8c68-020fde4238cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:15.661694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3a326-7bf9-4c3b-8c68-020fde4238cb service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:15.673139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1138/2257] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:15.694237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d47cf7-7dc7-481d-b074-3479f58a34d2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:15.775175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d47cf7-7dc7-481d-b074-3479f58a34d2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:15.780248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:15.781045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:15.781327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:15.781604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] PATH_INFO: `/v3/users/b1aca714eee24db9aaa274bb4d59a064` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:15.782881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1aca714eee24db9aaa274bb4d59a064)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:15.785176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:16.035154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1aca714eee24db9aaa274bb4d59a064'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:16.039523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1aca714eee24db9aaa274bb4d59a064'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:16.147942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1aca714eee24db9aaa274bb4d59a064'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:16.156535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1aca714eee24db9aaa274bb4d59a064'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:16.157282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1aca714eee24db9aaa274bb4d59a064'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:16.181704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1120/2258] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:15 2022] DELETE /identity/v3/users/b1aca714eee24db9aaa274bb4d59a064 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:16.189138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11127dc1-282b-4d18-94bf-0519e3b19278 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:16.189457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11127dc1-282b-4d18-94bf-0519e3b19278 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:16.189923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11127dc1-282b-4d18-94bf-0519e3b19278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:16.214448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-11127dc1-282b-4d18-94bf-0519e3b19278 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:16.215694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-11127dc1-282b-4d18-94bf-0519e3b19278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:16.288166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1139/2259] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:16.298746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:16.300194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:16.302673 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:16.361990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e1e4-2f1f-4a1b-b3a4-2d1f6783989e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:16.365784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384ff0d-b3aa-4b2d-b04b-1dfc8696b421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:16.366440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ff0d-b3aa-4b2d-b04b-1dfc8696b421 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:16.366907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ff0d-b3aa-4b2d-b04b-1dfc8696b421 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:16.367197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ff0d-b3aa-4b2d-b04b-1dfc8696b421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:16.379247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384ff0d-b3aa-4b2d-b04b-1dfc8696b421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:16.379247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384ff0d-b3aa-4b2d-b04b-1dfc8696b421 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:16.390140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1121/2260] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:16.854334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:16.907081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11127dc1-282b-4d18-94bf-0519e3b19278 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:16.908852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-11127dc1-282b-4d18-94bf-0519e3b19278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:16.928374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11127dc1-282b-4d18-94bf-0519e3b19278 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:16.939161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:16.943022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:16.943022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:16.943022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] PATH_INFO: `/v3/projects/63b96ecb614d47dd8645a940658c11b1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:16.943818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63b96ecb614d47dd8645a940658c11b1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:16.948058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:17.131795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63b96ecb614d47dd8645a940658c11b1', 'request_id': 'req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:17.140099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63b96ecb614d47dd8645a940658c11b1', 'request_id': 'req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:17.270897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63b96ecb614d47dd8645a940658c11b1', 'request_id': 'req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:17.329244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] The token cache is being invalidate because project 63b96ecb614d47dd8645a940658c11b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:17.329547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:17.330868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9eda1-de78-4bbb-a31e-c910ab29c70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:17.332547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1140/2261] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:16 2022] DELETE /identity/v3/projects/63b96ecb614d47dd8645a940658c11b1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:17.340408 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9772-928b-4f22-b2e0-3c5d9ebcc8fa None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:17.340699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9772-928b-4f22-b2e0-3c5d9ebcc8fa None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:17.340975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9772-928b-4f22-b2e0-3c5d9ebcc8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:17.368063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8abe9772-928b-4f22-b2e0-3c5d9ebcc8fa None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:17.369442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8abe9772-928b-4f22-b2e0-3c5d9ebcc8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:17.527409 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1122/2262] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:17.537134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a1740-0af5-4e0c-b7a8-e6dbb77a1154 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:17.537415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a1740-0af5-4e0c-b7a8-e6dbb77a1154 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:17.537890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a1740-0af5-4e0c-b7a8-e6dbb77a1154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:17.559329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-466a1740-0af5-4e0c-b7a8-e6dbb77a1154 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:17.561502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-466a1740-0af5-4e0c-b7a8-e6dbb77a1154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:17.570764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1141/2263] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:17.575554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe9772-928b-4f22-b2e0-3c5d9ebcc8fa None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:17.588033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe9772-928b-4f22-b2e0-3c5d9ebcc8fa None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:17.597101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:17.598111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:17.598573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:17.599017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:17.600631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:17.601091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:17.603782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:17.622501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1123/2264] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 04:06:17.628212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a1740-0af5-4e0c-b7a8-e6dbb77a1154 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:17.645786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a1740-0af5-4e0c-b7a8-e6dbb77a1154 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:17.651982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:17.652885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:17.653168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:17.653445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:17.654183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:17.656660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:17.798749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1142/2265] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:17 2022] POST /identity/v3/projects => generated 359 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:06:17.805310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:17.818211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6068c95-e04e-4781-a29a-e10ddaadd9ee admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:17.855447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:17.856922 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:17.898054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e179c-04f8-4c2c-bf19-ef9def8077a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:17.898889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e179c-04f8-4c2c-bf19-ef9def8077a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:17.899076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e179c-04f8-4c2c-bf19-ef9def8077a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:17.899257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e179c-04f8-4c2c-bf19-ef9def8077a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:17.899972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e179c-04f8-4c2c-bf19-ef9def8077a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:17.901912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e179c-04f8-4c2c-bf19-ef9def8077a8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:17.941660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90fb032-eb25-46d0-941c-08fad0c2830d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:17.947849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:17.948953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:17.949456 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:17.949956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:17.953228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:18.016854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:18.018349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:18.028546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1143/2266] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:06:18.126553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1124/2267] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:17 2022] POST /identity/v3/users => generated 280 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:06:18.130939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:18.143667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8413c-87a3-49d2-b655-5475232c66df service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:18.148617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4ab35-a187-48b0-8582-f6ad3d7c9b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:18.149240 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ab35-a187-48b0-8582-f6ad3d7c9b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:18.149684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ab35-a187-48b0-8582-f6ad3d7c9b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:18.149960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ab35-a187-48b0-8582-f6ad3d7c9b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:18.150688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4ab35-a187-48b0-8582-f6ad3d7c9b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:18.152356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4ab35-a187-48b0-8582-f6ad3d7c9b11 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:18.163708 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1144/2268] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:18.170292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e179c-04f8-4c2c-bf19-ef9def8077a8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:18.181224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e179c-04f8-4c2c-bf19-ef9def8077a8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:18.189715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d69b2-abe0-4035-b233-8e76585662ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:18.190940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d69b2-abe0-4035-b233-8e76585662ef admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:18.191343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d69b2-abe0-4035-b233-8e76585662ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:18.191639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d69b2-abe0-4035-b233-8e76585662ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:18.192555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d69b2-abe0-4035-b233-8e76585662ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:18.194503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d69b2-abe0-4035-b233-8e76585662ef admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:18.206617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1125/2269] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:18.213615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4ab35-a187-48b0-8582-f6ad3d7c9b11 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:18.224789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4ab35-a187-48b0-8582-f6ad3d7c9b11 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:18.230131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:18.231319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:18.231680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:18.232171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] PATH_INFO: `/v3/projects/a7dea088b9984118ae412799d971a8ba/users/88ad4e5b0b014049a1fc3d5cfe109c2b/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:18.261036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7dea088b9984118ae412799d971a8ba, user_id=88ad4e5b0b014049a1fc3d5cfe109c2b, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:18.266734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:18.425633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:18.429657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1145/2270] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:18 2022] PUT /identity/v3/projects/a7dea088b9984118ae412799d971a8ba/users/88ad4e5b0b014049a1fc3d5cfe109c2b/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:18.451340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73395-c470-4bf2-ae8a-515190a045be None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:18.451340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73395-c470-4bf2-ae8a-515190a045be None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:18.451340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73395-c470-4bf2-ae8a-515190a045be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:18.490759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2fa73395-c470-4bf2-ae8a-515190a045be None None] MFA Rules not processed for user `88ad4e5b0b014049a1fc3d5cfe109c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:18.492321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2fa73395-c470-4bf2-ae8a-515190a045be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:18.594684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1126/2271] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:18 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:18.652623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:18.657362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:18.813170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01363a7-ca0d-418e-a599-54cb5627a52b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:18.832517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044bb8d-e041-4d13-9425-d9c79095c3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:18.836584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044bb8d-e041-4d13-9425-d9c79095c3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:18.838409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044bb8d-e041-4d13-9425-d9c79095c3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:18.839202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044bb8d-e041-4d13-9425-d9c79095c3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:18.858026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044bb8d-e041-4d13-9425-d9c79095c3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:18.860663 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044bb8d-e041-4d13-9425-d9c79095c3e6 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:18.878649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1146/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:18 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:18.903103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa73395-c470-4bf2-ae8a-515190a045be None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:18.985951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:19.052332 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa73395-c470-4bf2-ae8a-515190a045be None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:19.058281 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:19.059418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:19.059819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:19.060418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] PATH_INFO: `/v3/users/88ad4e5b0b014049a1fc3d5cfe109c2b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:19.062009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] RBAC: Authorizing `identity:delete_user(user_id=88ad4e5b0b014049a1fc3d5cfe109c2b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:19.064730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:19.195489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:19.314092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88ad4e5b0b014049a1fc3d5cfe109c2b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:19.315094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88ad4e5b0b014049a1fc3d5cfe109c2b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:19.390694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88ad4e5b0b014049a1fc3d5cfe109c2b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:19.402831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88ad4e5b0b014049a1fc3d5cfe109c2b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:19.403855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88ad4e5b0b014049a1fc3d5cfe109c2b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:19.424094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1127/2273] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:18 2022] DELETE /identity/v3/users/88ad4e5b0b014049a1fc3d5cfe109c2b => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:19.431489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ac69-5bd5-4180-9169-2c923ec903bd None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:19.432951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ac69-5bd5-4180-9169-2c923ec903bd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:19.432951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ac69-5bd5-4180-9169-2c923ec903bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:19.463873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9d61ac69-5bd5-4180-9169-2c923ec903bd None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:19.466636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9d61ac69-5bd5-4180-9169-2c923ec903bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:19.555639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1147/2274] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:19.567078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:19.568743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:19.666289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad21f8-3be7-41d1-9efe-eaadad23963e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:19.670080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:19.670875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:19.671023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:19.671333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:19.688576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:19.691296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:19.710609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1128/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:06:20.140566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:20.285465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ac69-5bd5-4180-9169-2c923ec903bd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:20.288431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9d61ac69-5bd5-4180-9169-2c923ec903bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:20.316674 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ac69-5bd5-4180-9169-2c923ec903bd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:20.321915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:20.333313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:20.333313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:20.333313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] PATH_INFO: `/v3/projects/a7dea088b9984118ae412799d971a8ba` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:20.333313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7dea088b9984118ae412799d971a8ba)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:20.333313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:20.415566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:20.417586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:20.449943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:20.455751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7dea088b9984118ae412799d971a8ba', 'request_id': 'req-23625950-6c50-41eb-9973-f926e5a28cf1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:20.462035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7dea088b9984118ae412799d971a8ba', 'request_id': 'req-23625950-6c50-41eb-9973-f926e5a28cf1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:20.488712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788132fa-e89c-4b5f-b3b2-025688d2fecb service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:20.553016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a2d86-1017-4e82-8a59-b8c646aec996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:20.554091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2d86-1017-4e82-8a59-b8c646aec996 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:20.554651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2d86-1017-4e82-8a59-b8c646aec996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:20.554991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2d86-1017-4e82-8a59-b8c646aec996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:20.558069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-984a2d86-1017-4e82-8a59-b8c646aec996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:20.596756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7dea088b9984118ae412799d971a8ba', 'request_id': 'req-23625950-6c50-41eb-9973-f926e5a28cf1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:20.617251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] The token cache is being invalidate because project a7dea088b9984118ae412799d971a8ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:20.617755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:20.618801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-23625950-6c50-41eb-9973-f926e5a28cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:20.621726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1148/2276] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:20 2022] DELETE /identity/v3/projects/a7dea088b9984118ae412799d971a8ba => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:20.630908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10490a5a-c40c-481f-807f-89d8f5772c75 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:20.631453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10490a5a-c40c-481f-807f-89d8f5772c75 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:20.632197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10490a5a-c40c-481f-807f-89d8f5772c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:20.645902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a2d86-1017-4e82-8a59-b8c646aec996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:20.647691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a2d86-1017-4e82-8a59-b8c646aec996 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:20.659886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1129/2277] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:20.662691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-10490a5a-c40c-481f-807f-89d8f5772c75 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:20.667021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:20.667974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-10490a5a-c40c-481f-807f-89d8f5772c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:20.762324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1149/2278] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:20.770375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba49f-b00c-4f3d-a90a-478e969622dc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:20.770649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba49f-b00c-4f3d-a90a-478e969622dc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:20.770929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba49f-b00c-4f3d-a90a-478e969622dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:20.800037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-cdaba49f-b00c-4f3d-a90a-478e969622dc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:20.802121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cdaba49f-b00c-4f3d-a90a-478e969622dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:20.822005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1130/2279] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:20.823780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10490a5a-c40c-481f-807f-89d8f5772c75 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:20.839343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10490a5a-c40c-481f-807f-89d8f5772c75 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:20.845553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:20.846324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:20.846764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:20.847035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:20.848003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:20.848486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:20.850729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:20.866546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1150/2280] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:20.872383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba49f-b00c-4f3d-a90a-478e969622dc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:20.890628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba49f-b00c-4f3d-a90a-478e969622dc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:20.900608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf6090-be67-40ad-a31d-5bb657fdbf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:20.902390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6090-be67-40ad-a31d-5bb657fdbf4b admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:20.902795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6090-be67-40ad-a31d-5bb657fdbf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:20.903206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6090-be67-40ad-a31d-5bb657fdbf4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:20.904297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf6090-be67-40ad-a31d-5bb657fdbf4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:20.906706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf6090-be67-40ad-a31d-5bb657fdbf4b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:21.101312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1131/2281] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:20 2022] POST /identity/v3/projects => generated 383 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:21.107071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:21.119098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73f33b-893b-4b60-be17-2392ccf74060 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:21.174904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:21.223830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a79c7-8480-4706-be51-a59479e11ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:21.224535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a79c7-8480-4706-be51-a59479e11ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:21.224818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a79c7-8480-4706-be51-a59479e11ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:21.225100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a79c7-8480-4706-be51-a59479e11ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:21.225857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a79c7-8480-4706-be51-a59479e11ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:21.227911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a79c7-8480-4706-be51-a59479e11ae0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:21.316546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:21.392681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1151/2282] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:21 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:06:21.398895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf6090-be67-40ad-a31d-5bb657fdbf4b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:21.410967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf6090-be67-40ad-a31d-5bb657fdbf4b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:21.416208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28a702-4f0a-4e71-9c43-9b39cef3cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:21.420831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28a702-4f0a-4e71-9c43-9b39cef3cd9c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:21.420831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28a702-4f0a-4e71-9c43-9b39cef3cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:21.420831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28a702-4f0a-4e71-9c43-9b39cef3cd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:21.420831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28a702-4f0a-4e71-9c43-9b39cef3cd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:21.420831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28a702-4f0a-4e71-9c43-9b39cef3cd9c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:21.434172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1132/2283] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:21.438597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a79c7-8480-4706-be51-a59479e11ae0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:21.448971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a79c7-8480-4706-be51-a59479e11ae0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:21.454427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49b476-6f58-4854-a071-618814a1ee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:21.455054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49b476-6f58-4854-a071-618814a1ee02 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:21.455533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49b476-6f58-4854-a071-618814a1ee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:21.455815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49b476-6f58-4854-a071-618814a1ee02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:21.456552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49b476-6f58-4854-a071-618814a1ee02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:21.458165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49b476-6f58-4854-a071-618814a1ee02 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:21.470059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1152/2284] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:21.474671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28a702-4f0a-4e71-9c43-9b39cef3cd9c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:21.485301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28a702-4f0a-4e71-9c43-9b39cef3cd9c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:21.490686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:21.491643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:21.491931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:21.492244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] PATH_INFO: `/v3/projects/2940ca71c4b24b5facc194b61d748a11/users/abc2e04754884398afc999d423e2a711/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:21.510827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2940ca71c4b24b5facc194b61d748a11, user_id=abc2e04754884398afc999d423e2a711, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:21.514169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:21.644624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1133/2285] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:21 2022] PUT /identity/v3/projects/2940ca71c4b24b5facc194b61d748a11/users/abc2e04754884398afc999d423e2a711/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:21.650191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442d1d4-b771-4ca7-af32-bd38d72ca95a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:21.650504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442d1d4-b771-4ca7-af32-bd38d72ca95a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:21.650782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442d1d4-b771-4ca7-af32-bd38d72ca95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:21.670809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f442d1d4-b771-4ca7-af32-bd38d72ca95a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:21.672097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f442d1d4-b771-4ca7-af32-bd38d72ca95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:21.762459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1153/2286] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:21.773895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:21.776506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:21.864941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e83966-5e6d-4271-a74c-a9f1b68ee94a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:21.871041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd2abe-fb88-4d2e-abf4-8556d6f45f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:21.873822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd2abe-fb88-4d2e-abf4-8556d6f45f75 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:21.874106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd2abe-fb88-4d2e-abf4-8556d6f45f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:21.874411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd2abe-fb88-4d2e-abf4-8556d6f45f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:21.887697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd2abe-fb88-4d2e-abf4-8556d6f45f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:21.892373 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd2abe-fb88-4d2e-abf4-8556d6f45f75 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:21.914287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1134/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:06:22.480391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:22.707081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:23.223740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:24.184964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:24.358439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:24.631564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442d1d4-b771-4ca7-af32-bd38d72ca95a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:24.633173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f442d1d4-b771-4ca7-af32-bd38d72ca95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:24.648053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442d1d4-b771-4ca7-af32-bd38d72ca95a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:24.653212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:24.654355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:24.654776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:24.655124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] PATH_INFO: `/v3/users/712d3f65099c43c0a97548d4fc452da5` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:24.656687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] RBAC: Authorizing `identity:delete_user(user_id=712d3f65099c43c0a97548d4fc452da5)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:24.659426 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:24.996185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712d3f65099c43c0a97548d4fc452da5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:24.997130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712d3f65099c43c0a97548d4fc452da5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:24.998501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712d3f65099c43c0a97548d4fc452da5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:25.024919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712d3f65099c43c0a97548d4fc452da5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:25.040037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712d3f65099c43c0a97548d4fc452da5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:25.231430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1154/2288] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:24 2022] DELETE /identity/v3/users/712d3f65099c43c0a97548d4fc452da5 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:25.236464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a75ca1-02af-4fbf-b9b4-efcd8d9a5d9e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:25.236853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a75ca1-02af-4fbf-b9b4-efcd8d9a5d9e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:25.237142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a75ca1-02af-4fbf-b9b4-efcd8d9a5d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:25.268154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d7a75ca1-02af-4fbf-b9b4-efcd8d9a5d9e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:25.270818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d7a75ca1-02af-4fbf-b9b4-efcd8d9a5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:25.431798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1135/2289] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:25.467653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:25.470156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:25.708842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a92f04-42f4-4e14-8252-a6a43291b18d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:25.723381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:25.724888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:25.725601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:25.726287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:25.755250 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:25.758969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:25.792532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1155/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:06:26.188540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:26.397755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:26.526354 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a75ca1-02af-4fbf-b9b4-efcd8d9a5d9e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:26.528788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d7a75ca1-02af-4fbf-b9b4-efcd8d9a5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:26.545804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a75ca1-02af-4fbf-b9b4-efcd8d9a5d9e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:26.551796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:26.552916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:26.553190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:26.556500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] PATH_INFO: `/v3/projects/c039f6f3b515439992f7f7b0a39c5a4c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:26.556500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c039f6f3b515439992f7f7b0a39c5a4c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:26.556905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:26.772094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c039f6f3b515439992f7f7b0a39c5a4c', 'request_id': 'req-6e65006a-66e3-4313-a2f9-facd68a3db26'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:26.790372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c039f6f3b515439992f7f7b0a39c5a4c', 'request_id': 'req-6e65006a-66e3-4313-a2f9-facd68a3db26'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:26.820479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c039f6f3b515439992f7f7b0a39c5a4c', 'request_id': 'req-6e65006a-66e3-4313-a2f9-facd68a3db26'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:26.907862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:26.911097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:26.940260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b533a45-c92f-449d-8c81-ebe84fea3371 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:27.038426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] The token cache is being invalidate because project c039f6f3b515439992f7f7b0a39c5a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:27.039865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.048171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.053492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1136/2291] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:26 2022] DELETE /identity/v3/projects/c039f6f3b515439992f7f7b0a39c5a4c => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:27.059980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:27.068227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:27.108883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:27.111828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:27.112578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:27.113662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] PATH_INFO: `/v3/users/85e071aa9a6f453f96d1923327ae306f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:27.116696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e071aa9a6f453f96d1923327ae306f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:27.123321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:27.249871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65006a-66e3-4313-a2f9-facd68a3db26 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:27.262516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:27.264657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:27.265187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:27.265798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] PATH_INFO: `/v3/users/5f43bbacc8594156a568ae90334d665f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:27.268562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f43bbacc8594156a568ae90334d665f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:27.272937 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:27.478668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e071aa9a6f453f96d1923327ae306f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.486751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e071aa9a6f453f96d1923327ae306f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.486751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e071aa9a6f453f96d1923327ae306f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.511731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e071aa9a6f453f96d1923327ae306f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.528239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6a137a76-cb36-41a7-ae44-058a8f4b4564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e071aa9a6f453f96d1923327ae306f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.584541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f43bbacc8594156a568ae90334d665f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.585903 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f43bbacc8594156a568ae90334d665f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.660210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1156/2292] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:26 2022] DELETE /identity/v3/users/85e071aa9a6f453f96d1923327ae306f => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:27.673967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00b93-f7b9-492a-b06d-a15b92edf610 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:27.674327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00b93-f7b9-492a-b06d-a15b92edf610 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:27.674610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00b93-f7b9-492a-b06d-a15b92edf610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:27.698751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bef00b93-f7b9-492a-b06d-a15b92edf610 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:27.700033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bef00b93-f7b9-492a-b06d-a15b92edf610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:27.736243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f43bbacc8594156a568ae90334d665f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.744636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f43bbacc8594156a568ae90334d665f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.748318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f43bbacc8594156a568ae90334d665f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:27.772350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1137/2293] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:27 2022] DELETE /identity/v3/users/5f43bbacc8594156a568ae90334d665f => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:27.784676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1157/2294] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:27.796793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:27.798769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:27.875747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7000c48-036c-4d92-afb7-b172dcea828d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:27.882253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c251e-cae7-45dd-92c4-ae3e5c699488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:27.883507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c251e-cae7-45dd-92c4-ae3e5c699488 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:27.883960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c251e-cae7-45dd-92c4-ae3e5c699488 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:27.885788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c251e-cae7-45dd-92c4-ae3e5c699488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:28.018648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c251e-cae7-45dd-92c4-ae3e5c699488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:28.020953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c251e-cae7-45dd-92c4-ae3e5c699488 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:28.036096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1138/2295] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:28.941192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef00b93-f7b9-492a-b06d-a15b92edf610 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:28.943202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bef00b93-f7b9-492a-b06d-a15b92edf610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:28.970978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef00b93-f7b9-492a-b06d-a15b92edf610 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:28.979161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:28.980063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:28.980341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:28.984534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] PATH_INFO: `/v3/projects/e27c12f4053a43198eee47036133e9b4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:28.984534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27c12f4053a43198eee47036133e9b4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:28.984534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:29.197969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27c12f4053a43198eee47036133e9b4', 'request_id': 'req-974c9ce1-b7b2-45f6-bc81-f674af59915e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:29.208682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27c12f4053a43198eee47036133e9b4', 'request_id': 'req-974c9ce1-b7b2-45f6-bc81-f674af59915e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:29.285118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27c12f4053a43198eee47036133e9b4', 'request_id': 'req-974c9ce1-b7b2-45f6-bc81-f674af59915e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:29.326597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] The token cache is being invalidate because project e27c12f4053a43198eee47036133e9b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:29.332058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:29.333769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:29.337125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1158/2296] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:28 2022] DELETE /identity/v3/projects/e27c12f4053a43198eee47036133e9b4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:29.344809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ebe2e-1902-4d02-976f-c3734b7573fb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:29.346381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ebe2e-1902-4d02-976f-c3734b7573fb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:29.347305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ebe2e-1902-4d02-976f-c3734b7573fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:29.366248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:29.370044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:29.399111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:29.485192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-604ebe2e-1902-4d02-976f-c3734b7573fb None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:29.498260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-604ebe2e-1902-4d02-976f-c3734b7573fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:29.559845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c9ce1-b7b2-45f6-bc81-f674af59915e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:29.566429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:29.580367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:29.582242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:29.584258 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:29.584727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] PATH_INFO: `/v3/projects/a1a9301a62af442694ed54c69d58928f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:29.586853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a9301a62af442694ed54c69d58928f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:29.590764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:29.706526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1139/2297] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:29.709694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e86ff1-6d31-455c-bab7-efcf5512e5e0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:29.710353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e86ff1-6d31-455c-bab7-efcf5512e5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:29.711004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e86ff1-6d31-455c-bab7-efcf5512e5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:29.758897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f0e86ff1-6d31-455c-bab7-efcf5512e5e0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:29.761738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f0e86ff1-6d31-455c-bab7-efcf5512e5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:29.783463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a9301a62af442694ed54c69d58928f', 'request_id': 'req-0be0f522-ecbd-4158-aefd-cd88521be6c0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:29.793033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a9301a62af442694ed54c69d58928f', 'request_id': 'req-0be0f522-ecbd-4158-aefd-cd88521be6c0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:29.879688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1140/2298] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:29.883164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e86ff1-6d31-455c-bab7-efcf5512e5e0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:29.908962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e86ff1-6d31-455c-bab7-efcf5512e5e0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:29.923879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:29.926132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:29.926419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:29.926702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:29.931717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:29.932123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:29.936464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:29.956229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1141/2299] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:29.960702 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:29.969651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a9301a62af442694ed54c69d58928f', 'request_id': 'req-0be0f522-ecbd-4158-aefd-cd88521be6c0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:29.979537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f02d37-33fb-48e7-b55c-0aa9ea0d30dd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:29.999317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:30.001190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:30.004021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:30.004570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:30.005619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:30.010748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] The token cache is being invalidate because project a1a9301a62af442694ed54c69d58928f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:30.011180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:30.012599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:30.013154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:30.018224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1159/2300] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:29 2022] DELETE /identity/v3/projects/a1a9301a62af442694ed54c69d58928f => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:30.024571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:30.029772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:30.063173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0f522-ecbd-4158-aefd-cd88521be6c0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:30.083780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:30.085486 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:30.086004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:30.086852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] PATH_INFO: `/v3/users/e47bcf3cd7ff41399433cb2c0a186ade` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:30.092756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47bcf3cd7ff41399433cb2c0a186ade)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:30.099101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:30.177169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1142/2301] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:29 2022] POST /identity/v3/projects => generated 381 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:30.178214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:30.185522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:30.280759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d949cfd-3992-492c-8229-cfd02874c077 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:30.287208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:30.288024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:30.288431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:30.288866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:30.289810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:30.297534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:30.426959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1143/2302] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:30 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:30.427622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47bcf3cd7ff41399433cb2c0a186ade'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:30.427622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47bcf3cd7ff41399433cb2c0a186ade'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:30.428179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47bcf3cd7ff41399433cb2c0a186ade'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:30.429183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:30.431197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:30.449465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47bcf3cd7ff41399433cb2c0a186ade'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:30.464665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1e3222c3-281e-42de-bc52-d694c254ad77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47bcf3cd7ff41399433cb2c0a186ade'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:30.465239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39c2d6-7184-471d-8fce-62f76b0ec88d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:30.470397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bab15-9c1b-4537-a550-1ebbbb4cf3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:30.471425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bab15-9c1b-4537-a550-1ebbbb4cf3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:30.471858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bab15-9c1b-4537-a550-1ebbbb4cf3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:30.472278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bab15-9c1b-4537-a550-1ebbbb4cf3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:30.473178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bab15-9c1b-4537-a550-1ebbbb4cf3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:30.475296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bab15-9c1b-4537-a550-1ebbbb4cf3ce admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:30.495335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1144/2303] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:30.497144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bab15-9c1b-4537-a550-1ebbbb4cf3ce admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:30.518417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bab15-9c1b-4537-a550-1ebbbb4cf3ce admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:30.525471 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791286-1102-4769-8e0d-033cbe854a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:30.526723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791286-1102-4769-8e0d-033cbe854a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:30.527203 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791286-1102-4769-8e0d-033cbe854a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:30.527662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791286-1102-4769-8e0d-033cbe854a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:30.529150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af791286-1102-4769-8e0d-033cbe854a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:30.530999 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af791286-1102-4769-8e0d-033cbe854a5b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:30.547016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1145/2304] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:30.551921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791286-1102-4769-8e0d-033cbe854a5b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:30.566088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791286-1102-4769-8e0d-033cbe854a5b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:30.570269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1160/2305] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:30 2022] DELETE /identity/v3/users/e47bcf3cd7ff41399433cb2c0a186ade => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:30.578093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb994e-87b8-4b89-bd5a-51a96ceac4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:30.578093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb994e-87b8-4b89-bd5a-51a96ceac4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:30.578093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb994e-87b8-4b89-bd5a-51a96ceac4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:30.578093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb994e-87b8-4b89-bd5a-51a96ceac4d4 admin admin] PATH_INFO: `/v3/projects/53258e9776194b4d9652a06f4a6fdd4b/users/64c74e6e27c94e1b8f2d394194fffd00/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:30.603769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb994e-87b8-4b89-bd5a-51a96ceac4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53258e9776194b4d9652a06f4a6fdd4b, user_id=64c74e6e27c94e1b8f2d394194fffd00, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:30.608815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb994e-87b8-4b89-bd5a-51a96ceac4d4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:30.715971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1146/2306] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:30 2022] PUT /identity/v3/projects/53258e9776194b4d9652a06f4a6fdd4b/users/64c74e6e27c94e1b8f2d394194fffd00/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:30.738863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed554a1d-bc4f-473e-9c84-e5a58d8c5b1d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:30.739306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed554a1d-bc4f-473e-9c84-e5a58d8c5b1d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:30.739689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed554a1d-bc4f-473e-9c84-e5a58d8c5b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:30.801705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ed554a1d-bc4f-473e-9c84-e5a58d8c5b1d None None] MFA Rules not processed for user `64c74e6e27c94e1b8f2d394194fffd00`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:30.824144 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ed554a1d-bc4f-473e-9c84-e5a58d8c5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:30.928569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1161/2307] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:30.930915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb994e-87b8-4b89-bd5a-51a96ceac4d4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:31.088491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb994e-87b8-4b89-bd5a-51a96ceac4d4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:31.095516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:31.096532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:31.096965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:31.097506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:31.098439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:31.100892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:31.221394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1147/2308] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:30 2022] POST /identity/v3/projects => generated 381 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:31.227312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed554a1d-bc4f-473e-9c84-e5a58d8c5b1d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:31.243948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed554a1d-bc4f-473e-9c84-e5a58d8c5b1d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:31.370044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cc84f-dcc1-4faf-8cce-45fedc90475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:31.371803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc84f-dcc1-4faf-8cce-45fedc90475c admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:31.372591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc84f-dcc1-4faf-8cce-45fedc90475c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:31.373905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc84f-dcc1-4faf-8cce-45fedc90475c admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:31.375457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cc84f-dcc1-4faf-8cce-45fedc90475c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:31.379848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cc84f-dcc1-4faf-8cce-45fedc90475c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:31.540804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1162/2309] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:31 2022] POST /identity/v3/users => generated 297 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:31.541441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:31.545176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:31.552618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cc84f-dcc1-4faf-8cce-45fedc90475c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:31.605494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cc84f-dcc1-4faf-8cce-45fedc90475c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:31.620160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7298a0b-ffe0-43d7-8118-5eb0c79eb171 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:31.628641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b51bde-4368-4df6-8bb1-18c74d5dd0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:31.629350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b51bde-4368-4df6-8bb1-18c74d5dd0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:31.629641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b51bde-4368-4df6-8bb1-18c74d5dd0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:31.629915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b51bde-4368-4df6-8bb1-18c74d5dd0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:31.630670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b51bde-4368-4df6-8bb1-18c74d5dd0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:31.640533 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b51bde-4368-4df6-8bb1-18c74d5dd0fb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:31.642826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:31.644349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:31.644847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:31.647307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] PATH_INFO: `/v3/projects/00194ee4cc0948e0b603e63373126207` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:31.652103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00194ee4cc0948e0b603e63373126207)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:31.662052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:31.679560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1163/2310] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:31.680828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b51bde-4368-4df6-8bb1-18c74d5dd0fb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:31.716737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b51bde-4368-4df6-8bb1-18c74d5dd0fb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:31.735356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd5579-c4a9-4f03-8a9c-1ac3516a6059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:31.736268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5579-c4a9-4f03-8a9c-1ac3516a6059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:31.736546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5579-c4a9-4f03-8a9c-1ac3516a6059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:31.736819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5579-c4a9-4f03-8a9c-1ac3516a6059 admin admin] PATH_INFO: `/v3/projects/ff9232e8cd994cbcb18cbfd5af249d1b/users/722a9c00703f45679ce9f23f1e52b5a3/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:31.807816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd5579-c4a9-4f03-8a9c-1ac3516a6059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9232e8cd994cbcb18cbfd5af249d1b, user_id=722a9c00703f45679ce9f23f1e52b5a3, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:31.814437 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd5579-c4a9-4f03-8a9c-1ac3516a6059 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:31.910530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1164/2311] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:31 2022] PUT /identity/v3/projects/ff9232e8cd994cbcb18cbfd5af249d1b/users/722a9c00703f45679ce9f23f1e52b5a3/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:31.914926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd5579-c4a9-4f03-8a9c-1ac3516a6059 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:31.952946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00194ee4cc0948e0b603e63373126207', 'request_id': 'req-92b67049-cf68-4ee4-8dca-5147d2af2be4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:31.979037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00194ee4cc0948e0b603e63373126207', 'request_id': 'req-92b67049-cf68-4ee4-8dca-5147d2af2be4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:32.010434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00194ee4cc0948e0b603e63373126207', 'request_id': 'req-92b67049-cf68-4ee4-8dca-5147d2af2be4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:32.075124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd5579-c4a9-4f03-8a9c-1ac3516a6059 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:32.146222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] The token cache is being invalidate because project 00194ee4cc0948e0b603e63373126207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:32.148718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:32.154919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:32.161882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1148/2312] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:31 2022] DELETE /identity/v3/projects/00194ee4cc0948e0b603e63373126207 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:32.168962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:32.170992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:32.206010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:32.206887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:32.207262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:32.207753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:32.208731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:32.210685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:32.222692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:32.232790 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1165/2313] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:32.236293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:32.238870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:32.271493 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688c303-842f-4ffa-9455-a6974f6c0d4d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:32.287271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a70f2e-5f9a-474d-b1fa-dc52d090ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:32.288514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a70f2e-5f9a-474d-b1fa-dc52d090ebbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:32.288811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a70f2e-5f9a-474d-b1fa-dc52d090ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:32.289197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a70f2e-5f9a-474d-b1fa-dc52d090ebbf admin admin] PATH_INFO: `/v3/projects/ff9232e8cd994cbcb18cbfd5af249d1b/users/722a9c00703f45679ce9f23f1e52b5a3/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:32.298827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a70f2e-5f9a-474d-b1fa-dc52d090ebbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9232e8cd994cbcb18cbfd5af249d1b, user_id=722a9c00703f45679ce9f23f1e52b5a3, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:32.302454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a70f2e-5f9a-474d-b1fa-dc52d090ebbf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:32.309492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:32.329298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1166/2314] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:32 2022] PUT /identity/v3/projects/ff9232e8cd994cbcb18cbfd5af249d1b/users/722a9c00703f45679ce9f23f1e52b5a3/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:32.343733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acf0b5-2c84-4284-86bb-1bb1613699ac None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:32.344288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acf0b5-2c84-4284-86bb-1bb1613699ac None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:32.349023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acf0b5-2c84-4284-86bb-1bb1613699ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:32.408718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f7acf0b5-2c84-4284-86bb-1bb1613699ac None None] MFA Rules not processed for user `722a9c00703f45679ce9f23f1e52b5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:32.410253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f7acf0b5-2c84-4284-86bb-1bb1613699ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:32.465470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67049-cf68-4ee4-8dca-5147d2af2be4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:32.566332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1167/2315] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:32.590327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acf0b5-2c84-4284-86bb-1bb1613699ac None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:32.592748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f7acf0b5-2c84-4284-86bb-1bb1613699ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:32.648932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:32.651315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:32.652045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:32.652725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] PATH_INFO: `/v3/users/3fbf201a63ad4598989dd0309a67bba5` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:32.656297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fbf201a63ad4598989dd0309a67bba5)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:32.659960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:32.831662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acf0b5-2c84-4284-86bb-1bb1613699ac None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:32.843735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235579ee-fc47-45cf-8eb6-203b1bc0fa46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:32.844938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235579ee-fc47-45cf-8eb6-203b1bc0fa46 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:32.845480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235579ee-fc47-45cf-8eb6-203b1bc0fa46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:32.846037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235579ee-fc47-45cf-8eb6-203b1bc0fa46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:32.886898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235579ee-fc47-45cf-8eb6-203b1bc0fa46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:32.889207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235579ee-fc47-45cf-8eb6-203b1bc0fa46 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:32.910185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1168/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:06:32.935442 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fbf201a63ad4598989dd0309a67bba5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:32.938790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fbf201a63ad4598989dd0309a67bba5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:33.056673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fbf201a63ad4598989dd0309a67bba5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:33.070171 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fbf201a63ad4598989dd0309a67bba5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:33.071074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fbf201a63ad4598989dd0309a67bba5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:33.096248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8805f6-d06f-4217-97a1-32987c6832ca None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:33.096905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8805f6-d06f-4217-97a1-32987c6832ca None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:33.097623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8805f6-d06f-4217-97a1-32987c6832ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:33.131895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1149/2317] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:32 2022] DELETE /identity/v3/users/3fbf201a63ad4598989dd0309a67bba5 => generated 0 bytes in 966 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:33.141775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6d8805f6-d06f-4217-97a1-32987c6832ca None None] MFA Rules not processed for user `abc2e04754884398afc999d423e2a711`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:33.143098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6d8805f6-d06f-4217-97a1-32987c6832ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:33.213185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:33.215385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:33.265036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1169/2318] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:33.270842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8805f6-d06f-4217-97a1-32987c6832ca None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:33.273146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6d8805f6-d06f-4217-97a1-32987c6832ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:33.318001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d33038-e4ac-4067-a0d3-df994424bd3d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:33.323175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b081c7-540c-4593-ad06-782646c9479e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:33.324399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b081c7-540c-4593-ad06-782646c9479e service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:33.324682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b081c7-540c-4593-ad06-782646c9479e service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:33.324957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b081c7-540c-4593-ad06-782646c9479e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:33.327516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-17b081c7-540c-4593-ad06-782646c9479e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:33.362884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8805f6-d06f-4217-97a1-32987c6832ca None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:33.371370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea623763-a212-47d0-89ec-5b9f8b7cfc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:33.373052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea623763-a212-47d0-89ec-5b9f8b7cfc7e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:33.373703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea623763-a212-47d0-89ec-5b9f8b7cfc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:33.374200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea623763-a212-47d0-89ec-5b9f8b7cfc7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:33.375551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea623763-a212-47d0-89ec-5b9f8b7cfc7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:33.379394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea623763-a212-47d0-89ec-5b9f8b7cfc7e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:33.401567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b081c7-540c-4593-ad06-782646c9479e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:33.403255 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b081c7-540c-4593-ad06-782646c9479e service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:33.416504 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:33.423010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1150/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:33.452898 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836628e-601d-4ef9-944a-ac516f2d8a4f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:33.453644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836628e-601d-4ef9-944a-ac516f2d8a4f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:33.453644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836628e-601d-4ef9-944a-ac516f2d8a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:33.513522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e836628e-601d-4ef9-944a-ac516f2d8a4f None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:33.515176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e836628e-601d-4ef9-944a-ac516f2d8a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:33.589475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1170/2320] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:33 2022] POST /identity/v3/projects => generated 385 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:33.591459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea623763-a212-47d0-89ec-5b9f8b7cfc7e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:33.595264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1151/2321] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:06:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:33.695030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:33.752782 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836628e-601d-4ef9-944a-ac516f2d8a4f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:33.755106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e836628e-601d-4ef9-944a-ac516f2d8a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:33.952737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea623763-a212-47d0-89ec-5b9f8b7cfc7e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:33.961256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:33.962150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:33.962593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:33.963085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:33.964183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:33.979033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:34.026541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836628e-601d-4ef9-944a-ac516f2d8a4f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:34.126118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef786416-50fd-4500-a95e-9ec287efc7e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:34.127825 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef786416-50fd-4500-a95e-9ec287efc7e5 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:34.128855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef786416-50fd-4500-a95e-9ec287efc7e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:34.134021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef786416-50fd-4500-a95e-9ec287efc7e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:34.144218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1171/2322] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:33 2022] POST /identity/v3/users => generated 299 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:34.146930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:34.150170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:34.180950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cf8fa-4fe7-4da9-b78b-1ca9e613c1a8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:34.187721 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:34.188870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:34.189151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:34.189412 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] PATH_INFO: `/v3/projects/343648df590d40a1b8308f397a446dcf` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:34.190769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=343648df590d40a1b8308f397a446dcf)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:34.193135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:34.229390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef786416-50fd-4500-a95e-9ec287efc7e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:34.231609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef786416-50fd-4500-a95e-9ec287efc7e5 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:34.243138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1152/2323] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:06:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 04:06:34.244208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef786416-50fd-4500-a95e-9ec287efc7e5 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:34.258713 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef786416-50fd-4500-a95e-9ec287efc7e5 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:34.266413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82754e-9c39-440b-9b4a-771d388c18d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:34.267712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82754e-9c39-440b-9b4a-771d388c18d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:34.267712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82754e-9c39-440b-9b4a-771d388c18d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:34.268040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82754e-9c39-440b-9b4a-771d388c18d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:34.268891 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82754e-9c39-440b-9b4a-771d388c18d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:34.271586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82754e-9c39-440b-9b4a-771d388c18d7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:34.305483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1153/2324] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:34.306697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82754e-9c39-440b-9b4a-771d388c18d7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:34.322652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82754e-9c39-440b-9b4a-771d388c18d7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:34.329497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:34.330418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:34.330704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:34.330981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] PATH_INFO: `/v3/projects/8660d76489724a8f89de4df44cfd5bec/users/2f12518be45d4e3382554f5d5c7d291c/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:34.352766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8660d76489724a8f89de4df44cfd5bec, user_id=2f12518be45d4e3382554f5d5c7d291c, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:34.355723 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:34.491780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1154/2325] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:34 2022] PUT /identity/v3/projects/8660d76489724a8f89de4df44cfd5bec/users/2f12518be45d4e3382554f5d5c7d291c/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:34.493344 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:34.496505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:34.507424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '343648df590d40a1b8308f397a446dcf', 'request_id': 'req-f4674736-2d78-4ca0-9b55-80c80ab648c9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:34.538267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:34.540889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '343648df590d40a1b8308f397a446dcf', 'request_id': 'req-f4674736-2d78-4ca0-9b55-80c80ab648c9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:34.615021 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0f1e4-6d95-49b3-a3e1-1d8bb6ec611f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:34.686307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:34.687453 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:34.688051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:34.688407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] PATH_INFO: `/v3/users/64c74e6e27c94e1b8f2d394194fffd00` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:34.689899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c74e6e27c94e1b8f2d394194fffd00)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:34.698590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:34.705869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '343648df590d40a1b8308f397a446dcf', 'request_id': 'req-f4674736-2d78-4ca0-9b55-80c80ab648c9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:34.766987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] The token cache is being invalidate because project 343648df590d40a1b8308f397a446dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:34.767314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:34.772322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:34.774927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1172/2326] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:34 2022] DELETE /identity/v3/projects/343648df590d40a1b8308f397a446dcf => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:34.776102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:34.779344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:34.840701 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4674736-2d78-4ca0-9b55-80c80ab648c9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:34.859719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c61ef-d57c-468b-8b16-8add56b24025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:34.860357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c61ef-d57c-468b-8b16-8add56b24025 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:34.864585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c61ef-d57c-468b-8b16-8add56b24025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:34.865054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c61ef-d57c-468b-8b16-8add56b24025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:34.865874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c61ef-d57c-468b-8b16-8add56b24025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:34.876775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c61ef-d57c-468b-8b16-8add56b24025 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:34.928223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1173/2327] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:34.929811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd889-a558-44d6-96c2-4c9263ea5763 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:34.930107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd889-a558-44d6-96c2-4c9263ea5763 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:34.930383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd889-a558-44d6-96c2-4c9263ea5763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:34.963214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8ebcd889-a558-44d6-96c2-4c9263ea5763 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:34.964529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8ebcd889-a558-44d6-96c2-4c9263ea5763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:34.975963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1174/2328] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:34.977364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcd889-a558-44d6-96c2-4c9263ea5763 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:34.993507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcd889-a558-44d6-96c2-4c9263ea5763 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:35.002241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aea1a3-10fb-4101-bdb5-ab787268927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:35.003834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aea1a3-10fb-4101-bdb5-ab787268927d admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:35.004331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aea1a3-10fb-4101-bdb5-ab787268927d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:35.006867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aea1a3-10fb-4101-bdb5-ab787268927d admin admin] PATH_INFO: `/v3/projects/8660d76489724a8f89de4df44cfd5bec/users/2f12518be45d4e3382554f5d5c7d291c/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:35.007578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aea1a3-10fb-4101-bdb5-ab787268927d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8660d76489724a8f89de4df44cfd5bec, user_id=2f12518be45d4e3382554f5d5c7d291c, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:35.010798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aea1a3-10fb-4101-bdb5-ab787268927d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:35.035284 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1175/2329] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:34 2022] PUT /identity/v3/projects/8660d76489724a8f89de4df44cfd5bec/users/2f12518be45d4e3382554f5d5c7d291c/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:35.036929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69554926-fadc-4c80-b21b-48094cdcba00 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:35.037738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69554926-fadc-4c80-b21b-48094cdcba00 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:35.037738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69554926-fadc-4c80-b21b-48094cdcba00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:35.114341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-69554926-fadc-4c80-b21b-48094cdcba00 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:35.120836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-69554926-fadc-4c80-b21b-48094cdcba00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:35.213237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c74e6e27c94e1b8f2d394194fffd00'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:35.214313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c74e6e27c94e1b8f2d394194fffd00'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:35.332958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c74e6e27c94e1b8f2d394194fffd00'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:35.363664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c74e6e27c94e1b8f2d394194fffd00'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:35.387427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-875a5ef0-6b2f-4135-b246-7b90371e9676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c74e6e27c94e1b8f2d394194fffd00'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:35.550283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1176/2330] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:35.552806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69554926-fadc-4c80-b21b-48094cdcba00 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:35.555492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1155/2331] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:34 2022] DELETE /identity/v3/users/64c74e6e27c94e1b8f2d394194fffd00 => generated 0 bytes in 1063 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:35.566005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdfb4f-0aac-4f59-b333-fd4c05eaa919 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:35.566627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdfb4f-0aac-4f59-b333-fd4c05eaa919 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:35.566966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdfb4f-0aac-4f59-b333-fd4c05eaa919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:35.641085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69554926-fadc-4c80-b21b-48094cdcba00 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:35.705083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2abdfb4f-0aac-4f59-b333-fd4c05eaa919 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:35.711165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2abdfb4f-0aac-4f59-b333-fd4c05eaa919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:35.959930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1156/2332] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:35.974866 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:35.975397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:35.975681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:35.975957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:35.976749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:35.977062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:35.978826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:35.999616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1177/2333] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:36.000752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdfb4f-0aac-4f59-b333-fd4c05eaa919 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:36.014805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdfb4f-0aac-4f59-b333-fd4c05eaa919 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:36.016926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:36.019874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:36.045339 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a74f7-8e48-4be6-a9f0-8b3952be9b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:36.046037 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a74f7-8e48-4be6-a9f0-8b3952be9b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:36.046323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a74f7-8e48-4be6-a9f0-8b3952be9b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:36.046604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a74f7-8e48-4be6-a9f0-8b3952be9b5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:36.053823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a74f7-8e48-4be6-a9f0-8b3952be9b5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:36.074123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a74f7-8e48-4be6-a9f0-8b3952be9b5c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:36.203518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b3f0d-21b0-4185-9807-209ca6abd105 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:36.207088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:36.208676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa016b-cdd9-4be2-8959-0c2179a88822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:36.210164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa016b-cdd9-4be2-8959-0c2179a88822 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:36.210552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa016b-cdd9-4be2-8959-0c2179a88822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:36.210935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa016b-cdd9-4be2-8959-0c2179a88822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:36.224424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa016b-cdd9-4be2-8959-0c2179a88822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:36.226361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa016b-cdd9-4be2-8959-0c2179a88822 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:36.251665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1157/2334] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:35 2022] POST /identity/v3/projects => generated 389 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:06:36.252398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1178/2335] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:36.256088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a74f7-8e48-4be6-a9f0-8b3952be9b5c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:36.331719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30e44-da68-4609-9c98-4c5492309b87 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:36.332010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30e44-da68-4609-9c98-4c5492309b87 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:36.332289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30e44-da68-4609-9c98-4c5492309b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:36.358301 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-34e30e44-da68-4609-9c98-4c5492309b87 None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:36.374852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-34e30e44-da68-4609-9c98-4c5492309b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:36.390347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a74f7-8e48-4be6-a9f0-8b3952be9b5c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:36.395051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2663d-219e-49a7-8c43-eb23bf2c4e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:36.395780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2663d-219e-49a7-8c43-eb23bf2c4e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:36.396079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2663d-219e-49a7-8c43-eb23bf2c4e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:36.396623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2663d-219e-49a7-8c43-eb23bf2c4e01 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:36.397348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2663d-219e-49a7-8c43-eb23bf2c4e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:36.399588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2663d-219e-49a7-8c43-eb23bf2c4e01 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:36.482703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1179/2336] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 1 04:06:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:36.502242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e30e44-da68-4609-9c98-4c5492309b87 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:36.513269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-34e30e44-da68-4609-9c98-4c5492309b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:36.542515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:36.547156 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1158/2337] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:36 2022] POST /identity/v3/users => generated 295 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:06:36.551519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2663d-219e-49a7-8c43-eb23bf2c4e01 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:36.572184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2663d-219e-49a7-8c43-eb23bf2c4e01 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:36.593607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2755949-31c5-4eed-80d4-a17ad67dd330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:36.594079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755949-31c5-4eed-80d4-a17ad67dd330 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:36.594371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755949-31c5-4eed-80d4-a17ad67dd330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:36.594657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755949-31c5-4eed-80d4-a17ad67dd330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:36.595475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2755949-31c5-4eed-80d4-a17ad67dd330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:36.597147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2755949-31c5-4eed-80d4-a17ad67dd330 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:36.622068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1159/2338] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:36.629290 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2755949-31c5-4eed-80d4-a17ad67dd330 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:36.638000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2755949-31c5-4eed-80d4-a17ad67dd330 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:36.657066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162e459-5785-4db3-991f-a94f74dce53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:36.657779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e459-5785-4db3-991f-a94f74dce53b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:36.658063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e459-5785-4db3-991f-a94f74dce53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:36.658342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e459-5785-4db3-991f-a94f74dce53b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:36.659089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162e459-5785-4db3-991f-a94f74dce53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:36.672175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162e459-5785-4db3-991f-a94f74dce53b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:36.695853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1160/2339] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:36.702975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162e459-5785-4db3-991f-a94f74dce53b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:36.719474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162e459-5785-4db3-991f-a94f74dce53b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:36.731105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:36.732705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:36.733033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:36.736507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] PATH_INFO: `/v3/projects/d94f637cad9942a89fb3c98951070345/users/e674128711f3457c9e4156c6edf24edc/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:36.776628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e30e44-da68-4609-9c98-4c5492309b87 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:36.792290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f65e8-e4ef-46a8-8b94-08307959fb82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:36.793405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f65e8-e4ef-46a8-8b94-08307959fb82 service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:36.793910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f65e8-e4ef-46a8-8b94-08307959fb82 service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:36.795268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f65e8-e4ef-46a8-8b94-08307959fb82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:36.822987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d94f637cad9942a89fb3c98951070345, user_id=e674128711f3457c9e4156c6edf24edc, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:36.826654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f65e8-e4ef-46a8-8b94-08307959fb82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:36.831661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f65e8-e4ef-46a8-8b94-08307959fb82 service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:36.844511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:36.875296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1180/2340] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:06:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:06:36.993912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1161/2341] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:36 2022] PUT /identity/v3/projects/d94f637cad9942a89fb3c98951070345/users/e674128711f3457c9e4156c6edf24edc/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:37.016560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d982ba-8b0f-4616-83c8-62b2dab710e0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:37.017135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d982ba-8b0f-4616-83c8-62b2dab710e0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:37.017572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d982ba-8b0f-4616-83c8-62b2dab710e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:37.100782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-22d982ba-8b0f-4616-83c8-62b2dab710e0 None None] MFA Rules not processed for user `e674128711f3457c9e4156c6edf24edc`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:37.105442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-22d982ba-8b0f-4616-83c8-62b2dab710e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:37.314351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1181/2342] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:37.329411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:37.331507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:37.491323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac8912-d172-4d7f-b917-b4bd8a0d3657 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:37.498670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e61343-9ebf-4c3a-8ecb-724ce28ea761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:37.499945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e61343-9ebf-4c3a-8ecb-724ce28ea761 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:37.501434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e61343-9ebf-4c3a-8ecb-724ce28ea761 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:37.501919 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e61343-9ebf-4c3a-8ecb-724ce28ea761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:37.519394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e61343-9ebf-4c3a-8ecb-724ce28ea761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:37.521913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e61343-9ebf-4c3a-8ecb-724ce28ea761 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:37.522427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d982ba-8b0f-4616-83c8-62b2dab710e0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:37.524846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-22d982ba-8b0f-4616-83c8-62b2dab710e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:37.546697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1162/2343] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:37.757841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d982ba-8b0f-4616-83c8-62b2dab710e0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:37.764736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:37.766299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:37.766827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:37.767290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] PATH_INFO: `/v3/projects/53258e9776194b4d9652a06f4a6fdd4b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:37.775696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=53258e9776194b4d9652a06f4a6fdd4b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:37.775696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:38.007335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53258e9776194b4d9652a06f4a6fdd4b', 'request_id': 'req-c844ac4e-bdba-464d-ba29-9719b6454e0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:38.023828 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53258e9776194b4d9652a06f4a6fdd4b', 'request_id': 'req-c844ac4e-bdba-464d-ba29-9719b6454e0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:38.150738 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53258e9776194b4d9652a06f4a6fdd4b', 'request_id': 'req-c844ac4e-bdba-464d-ba29-9719b6454e0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:38.190300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] The token cache is being invalidate because project 53258e9776194b4d9652a06f4a6fdd4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:38.190805 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:38.192695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:38.198535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1182/2344] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:37 2022] DELETE /identity/v3/projects/53258e9776194b4d9652a06f4a6fdd4b => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:06:38.199686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:38.202380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42da272-2046-43a4-bd00-92333c6fbe70 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:38.203768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42da272-2046-43a4-bd00-92333c6fbe70 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:38.204887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42da272-2046-43a4-bd00-92333c6fbe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:38.206882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:38.326783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c42da272-2046-43a4-bd00-92333c6fbe70 None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:38.334244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c42da272-2046-43a4-bd00-92333c6fbe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:38.456097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844ac4e-bdba-464d-ba29-9719b6454e0b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:38.461516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:38.462705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:38.473176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:38.473176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] PATH_INFO: `/v3/users/722a9c00703f45679ce9f23f1e52b5a3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:38.473176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=722a9c00703f45679ce9f23f1e52b5a3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:38.473176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:38.484559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1163/2345] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:06:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:38.486353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e634db-f69b-48a6-9b47-d8f5f32b312c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:38.486654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e634db-f69b-48a6-9b47-d8f5f32b312c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:38.486933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e634db-f69b-48a6-9b47-d8f5f32b312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:38.510440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d1e634db-f69b-48a6-9b47-d8f5f32b312c None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:38.511757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d1e634db-f69b-48a6-9b47-d8f5f32b312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:38.521227 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1164/2346] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:06:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:38.522099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e634db-f69b-48a6-9b47-d8f5f32b312c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:38.523538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d1e634db-f69b-48a6-9b47-d8f5f32b312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:38.649173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e634db-f69b-48a6-9b47-d8f5f32b312c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:38.655569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87f683-fe42-4045-a57f-784b110e8932 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:38.656075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87f683-fe42-4045-a57f-784b110e8932 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:38.656356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87f683-fe42-4045-a57f-784b110e8932 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:38.656662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87f683-fe42-4045-a57f-784b110e8932 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:38.667527 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87f683-fe42-4045-a57f-784b110e8932 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:38.670016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87f683-fe42-4045-a57f-784b110e8932 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:38.679802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1165/2347] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:06:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:06:38.680695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87f683-fe42-4045-a57f-784b110e8932 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:38.694860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87f683-fe42-4045-a57f-784b110e8932 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:38.715993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91b9ae-92cb-40ef-831a-e5493bf58703 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:38.716678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91b9ae-92cb-40ef-831a-e5493bf58703 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:38.716959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91b9ae-92cb-40ef-831a-e5493bf58703 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:38.717239 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91b9ae-92cb-40ef-831a-e5493bf58703 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:38.745104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91b9ae-92cb-40ef-831a-e5493bf58703 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:38.749143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91b9ae-92cb-40ef-831a-e5493bf58703 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:38.781707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1166/2348] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:06:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:06:38.927572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '722a9c00703f45679ce9f23f1e52b5a3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:38.932525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '722a9c00703f45679ce9f23f1e52b5a3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:38.934570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '722a9c00703f45679ce9f23f1e52b5a3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:38.994633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '722a9c00703f45679ce9f23f1e52b5a3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:39.026454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-48dde740-a8dd-4144-bfe0-f89fc6982bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '722a9c00703f45679ce9f23f1e52b5a3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:39.221624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1183/2349] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:38 2022] DELETE /identity/v3/users/722a9c00703f45679ce9f23f1e52b5a3 => generated 0 bytes in 1023 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:39.846349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:40.009291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:40.372694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91b9ae-92cb-40ef-831a-e5493bf58703 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:40.409881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91b9ae-92cb-40ef-831a-e5493bf58703 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:40.602605 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2279-ccda-4293-8317-93bc59150258 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:40.603760 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2279-ccda-4293-8317-93bc59150258 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:40.604264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2279-ccda-4293-8317-93bc59150258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:40.632909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fe3e2279-ccda-4293-8317-93bc59150258 None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:40.634867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fe3e2279-ccda-4293-8317-93bc59150258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:40.641254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:40.642753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:40.643611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:40.644377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] PATH_INFO: `/v3/projects/ff9232e8cd994cbcb18cbfd5af249d1b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:40.647456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9232e8cd994cbcb18cbfd5af249d1b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:40.651817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:40.884335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1184/2350] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 1 04:06:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:40.923254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e2279-ccda-4293-8317-93bc59150258 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:40.928180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fe3e2279-ccda-4293-8317-93bc59150258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:41.043548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9232e8cd994cbcb18cbfd5af249d1b', 'request_id': 'req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:41.071347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9232e8cd994cbcb18cbfd5af249d1b', 'request_id': 'req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:41.114543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9232e8cd994cbcb18cbfd5af249d1b', 'request_id': 'req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:41.253362 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] The token cache is being invalidate because project ff9232e8cd994cbcb18cbfd5af249d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:41.255687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:41.259054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7f76fb5a-fd18-4b9e-9eb6-771c0de642f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:41.292082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1167/2351] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:40 2022] DELETE /identity/v3/projects/ff9232e8cd994cbcb18cbfd5af249d1b => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:41.298287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e2279-ccda-4293-8317-93bc59150258 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:41.300961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58501523-60cc-49a5-940e-8c3e8cfa3901 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:41.308566 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58501523-60cc-49a5-940e-8c3e8cfa3901 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:41.309011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58501523-60cc-49a5-940e-8c3e8cfa3901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:41.439776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-58501523-60cc-49a5-940e-8c3e8cfa3901 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:41.443865 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-58501523-60cc-49a5-940e-8c3e8cfa3901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:41.621637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:41.623723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:41.624127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:41.624626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:41.627045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:41.723141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1168/2352] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:41.726073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252d0f9-d1d7-4b23-ba9d-8a0b510ebab7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:41.726073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252d0f9-d1d7-4b23-ba9d-8a0b510ebab7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:41.726346 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252d0f9-d1d7-4b23-ba9d-8a0b510ebab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:41.726937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:41.729483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:41.741522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1185/2353] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:06:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 833 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:06:41.752759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a252d0f9-d1d7-4b23-ba9d-8a0b510ebab7 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:41.754047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a252d0f9-d1d7-4b23-ba9d-8a0b510ebab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:41.805776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:41.808041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1169/2354] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:41.858095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade74069-481e-47c6-87a0-491360778a78 service swift] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:41.863152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:41.869779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:41.870068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:41.870394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:41.872877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:41.874761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:41.876540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:41.901148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252d0f9-d1d7-4b23-ba9d-8a0b510ebab7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:41.912331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252d0f9-d1d7-4b23-ba9d-8a0b510ebab7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:41.913047 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1186/2355] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:41.925537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69c749-7207-4fac-9975-381148621690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:41.931319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69c749-7207-4fac-9975-381148621690 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:41.931922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69c749-7207-4fac-9975-381148621690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:41.932426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69c749-7207-4fac-9975-381148621690 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:41.933464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69c749-7207-4fac-9975-381148621690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:41.936478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69c749-7207-4fac-9975-381148621690 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:42.586419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1170/2356] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:41 2022] POST /identity/v3/projects => generated 377 bytes in 686 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:06:42.594910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:42.624774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da4792-0c57-4d1d-923e-ff3af8fc1644 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:42.820730 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3d022-1d18-41af-bdc9-877abad46dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:42.822570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3d022-1d18-41af-bdc9-877abad46dce admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:42.823123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3d022-1d18-41af-bdc9-877abad46dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:42.823706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3d022-1d18-41af-bdc9-877abad46dce admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:42.824710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3d022-1d18-41af-bdc9-877abad46dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:42.828289 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3d022-1d18-41af-bdc9-877abad46dce admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:43.077484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:43.084193 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1187/2357] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:42 2022] POST /identity/v3/users => generated 289 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:43.088829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69c749-7207-4fac-9975-381148621690 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:43.108350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69c749-7207-4fac-9975-381148621690 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:43.139451 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfd58a-8643-4fe1-be0a-1da6a26d4f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:43.140550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd58a-8643-4fe1-be0a-1da6a26d4f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:43.140839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd58a-8643-4fe1-be0a-1da6a26d4f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:43.141117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd58a-8643-4fe1-be0a-1da6a26d4f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:43.141932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfd58a-8643-4fe1-be0a-1da6a26d4f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:43.151911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfd58a-8643-4fe1-be0a-1da6a26d4f4d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:43.183631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3d022-1d18-41af-bdc9-877abad46dce admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:43.186421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1171/2358] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:43.203509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3d022-1d18-41af-bdc9-877abad46dce admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:43.218483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265680a-34de-4801-aa51-b5110031a17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:43.225811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265680a-34de-4801-aa51-b5110031a17c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:43.225811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265680a-34de-4801-aa51-b5110031a17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:43.225811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265680a-34de-4801-aa51-b5110031a17c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:43.225811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265680a-34de-4801-aa51-b5110031a17c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:43.225811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265680a-34de-4801-aa51-b5110031a17c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:43.238609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1188/2359] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:43.242443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfd58a-8643-4fe1-be0a-1da6a26d4f4d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:43.280736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfd58a-8643-4fe1-be0a-1da6a26d4f4d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:43.286640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:43.288397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:43.289086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:43.289527 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] PATH_INFO: `/v3/projects/922d87b0a9884af798e48d5035a98e48/users/c3a66e07a127464c8c12fd6111f20369/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:43.331603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=922d87b0a9884af798e48d5035a98e48, user_id=c3a66e07a127464c8c12fd6111f20369, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:43.340526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:43.469040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1172/2360] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:43 2022] PUT /identity/v3/projects/922d87b0a9884af798e48d5035a98e48/users/c3a66e07a127464c8c12fd6111f20369/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:43.478498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d1af8-c9bd-4001-bce5-640d67d83c14 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:43.478902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d1af8-c9bd-4001-bce5-640d67d83c14 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:43.479457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d1af8-c9bd-4001-bce5-640d67d83c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:43.567953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a91d1af8-c9bd-4001-bce5-640d67d83c14 None None] MFA Rules not processed for user `c3a66e07a127464c8c12fd6111f20369`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:43.570569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a91d1af8-c9bd-4001-bce5-640d67d83c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:43.843327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1189/2361] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:43.910862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:43.915731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:44.321603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7eeda-2663-41ce-9a0f-38b4fd1cd63f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:44.342953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:44.345401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:44.348120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:44.349718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:44.371006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:44.374593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:44.393977 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1173/2362] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:44.993883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:46.373951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146fc36-bc19-4428-94b6-64cfd3582e80 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:46.375860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146fc36-bc19-4428-94b6-64cfd3582e80 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:46.376632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146fc36-bc19-4428-94b6-64cfd3582e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:46.408587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1146fc36-bc19-4428-94b6-64cfd3582e80 None None] MFA Rules not processed for user `8622ec5212c84d87b9b7c8f725012a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:46.413018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1146fc36-bc19-4428-94b6-64cfd3582e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:46.593553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1190/2363] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:47.085334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:47.198163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:47.385891 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:47.388984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:47.595153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5396176-fd4b-40d7-84b8-92f0983e5a47 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:47.609140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:47.610787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:47.612241 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:47.612777 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] PATH_INFO: `/v3/users/d6515163518749d3bbe6e3eeda0a4019` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:47.615454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6515163518749d3bbe6e3eeda0a4019)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:47.625640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:48.028801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6515163518749d3bbe6e3eeda0a4019'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:48.030214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6515163518749d3bbe6e3eeda0a4019'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:48.190226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6515163518749d3bbe6e3eeda0a4019'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:48.227519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6515163518749d3bbe6e3eeda0a4019'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:48.232034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6515163518749d3bbe6e3eeda0a4019'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:48.303324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1174/2364] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:06:47 2022] DELETE /identity/v3/users/d6515163518749d3bbe6e3eeda0a4019 => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:48.303928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb184f-a791-4b60-9d30-729060269375 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:48.304885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb184f-a791-4b60-9d30-729060269375 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:48.305408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb184f-a791-4b60-9d30-729060269375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:48.365324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-36cb184f-a791-4b60-9d30-729060269375 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:48.367062 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-36cb184f-a791-4b60-9d30-729060269375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:48.573025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1191/2365] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:48.582878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:48.586980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:48.752980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953ad83-6cff-4379-921f-2c7697c61aa5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:48.759068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:48.760333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:48.761170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:48.761559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:48.779106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:48.780788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:48.794099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1175/2366] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:49.277617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b714e1-23a5-4c1b-8c18-f4aee01e23b8 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:49.278887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b714e1-23a5-4c1b-8c18-f4aee01e23b8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:49.279769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b714e1-23a5-4c1b-8c18-f4aee01e23b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:49.306151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:49.308589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:49.321263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-97b714e1-23a5-4c1b-8c18-f4aee01e23b8 None None] MFA Rules not processed for user `2f12518be45d4e3382554f5d5c7d291c`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:49.322851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-97b714e1-23a5-4c1b-8c18-f4aee01e23b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:49.352730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693bcff-3f07-4828-81e2-3058b16e1f90 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:49.385963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:49.388238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:49.391712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:49.393784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] PATH_INFO: `/v3/projects/12c54764cae34c12b35a6939f96aad5c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:49.397379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c54764cae34c12b35a6939f96aad5c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:49.404745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:49.428620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:49.470640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:49.623409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1192/2367] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:49.636230 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b714e1-23a5-4c1b-8c18-f4aee01e23b8 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:49.638192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-97b714e1-23a5-4c1b-8c18-f4aee01e23b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:49.710569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c54764cae34c12b35a6939f96aad5c', 'request_id': 'req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:49.735092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c54764cae34c12b35a6939f96aad5c', 'request_id': 'req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:49.744274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c54764cae34c12b35a6939f96aad5c', 'request_id': 'req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:49.762215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b714e1-23a5-4c1b-8c18-f4aee01e23b8 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:49.840110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] The token cache is being invalidate because project 12c54764cae34c12b35a6939f96aad5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:06:49.840415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:49.841982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904e3c7-efd4-4cac-a163-d43f0424433d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8327d9916424843b2a67cbdd856ad37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:49.842869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e3c7-efd4-4cac-a163-d43f0424433d service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:49.843453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e3c7-efd4-4cac-a163-d43f0424433d service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:49.843841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e3c7-efd4-4cac-a163-d43f0424433d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:49.845407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d6bc8d3b-3e08-4530-8f6f-f1f908d46d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:06:49.846468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1904e3c7-efd4-4cac-a163-d43f0424433d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:49.849059 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1176/2368] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:06:49 2022] DELETE /identity/v3/projects/12c54764cae34c12b35a6939f96aad5c => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:49.855015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2617799-46e6-4cba-a582-3f2f27f8077d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:49.855347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2617799-46e6-4cba-a582-3f2f27f8077d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:49.855641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2617799-46e6-4cba-a582-3f2f27f8077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:49.935547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e2617799-46e6-4cba-a582-3f2f27f8077d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:49.937162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e2617799-46e6-4cba-a582-3f2f27f8077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:49.957962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904e3c7-efd4-4cac-a163-d43f0424433d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:49.960239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904e3c7-efd4-4cac-a163-d43f0424433d service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:49.973215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1193/2369] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:50.081154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1177/2370] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:50.089159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c87a62-dbe8-4f3c-a31a-7691c77c1c0f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:50.089662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c87a62-dbe8-4f3c-a31a-7691c77c1c0f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:50.090025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c87a62-dbe8-4f3c-a31a-7691c77c1c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:50.118691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-14c87a62-dbe8-4f3c-a31a-7691c77c1c0f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:50.120298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-14c87a62-dbe8-4f3c-a31a-7691c77c1c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:50.141834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1194/2371] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:50.144740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2617799-46e6-4cba-a582-3f2f27f8077d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:50.158531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2617799-46e6-4cba-a582-3f2f27f8077d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:50.165619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8449-acfd-4a2f-9d05-45eeb024d0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:50.166466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8449-acfd-4a2f-9d05-45eeb024d0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:50.166853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8449-acfd-4a2f-9d05-45eeb024d0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:50.167256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8449-acfd-4a2f-9d05-45eeb024d0c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:50.168125 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8449-acfd-4a2f-9d05-45eeb024d0c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:06:50.168542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8449-acfd-4a2f-9d05-45eeb024d0c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:50.172708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8449-acfd-4a2f-9d05-45eeb024d0c0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:50.195664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1178/2372] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:06:50.196783 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c87a62-dbe8-4f3c-a31a-7691c77c1c0f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:50.225190 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c87a62-dbe8-4f3c-a31a-7691c77c1c0f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:50.240428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944382-8756-4ec4-9d4a-abba790d473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:50.241165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944382-8756-4ec4-9d4a-abba790d473d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:50.241450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944382-8756-4ec4-9d4a-abba790d473d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:50.243935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944382-8756-4ec4-9d4a-abba790d473d admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:50.244717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944382-8756-4ec4-9d4a-abba790d473d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:50.252327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944382-8756-4ec4-9d4a-abba790d473d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:50.323244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4564c-a27b-4cdd-8709-fc0ccf2c54ff None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:50.324371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4564c-a27b-4cdd-8709-fc0ccf2c54ff None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:50.324666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4564c-a27b-4cdd-8709-fc0ccf2c54ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:50.349072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-aad4564c-a27b-4cdd-8709-fc0ccf2c54ff None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:50.350774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aad4564c-a27b-4cdd-8709-fc0ccf2c54ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:50.783500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1179/2373] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:06:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:06:50.784987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1195/2374] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:06:50 2022] POST /identity/v3/projects => generated 381 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:50.785385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4564c-a27b-4cdd-8709-fc0ccf2c54ff None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:50.785385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aad4564c-a27b-4cdd-8709-fc0ccf2c54ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:50.785623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944382-8756-4ec4-9d4a-abba790d473d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:50.785724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4564c-a27b-4cdd-8709-fc0ccf2c54ff None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:50.871334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944382-8756-4ec4-9d4a-abba790d473d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:50.877129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2513e40-57c7-4238-bcb3-dee007b3fa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:50.877890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513e40-57c7-4238-bcb3-dee007b3fa32 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:50.878173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513e40-57c7-4238-bcb3-dee007b3fa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:50.878623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513e40-57c7-4238-bcb3-dee007b3fa32 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:50.879491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2513e40-57c7-4238-bcb3-dee007b3fa32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:50.881606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2513e40-57c7-4238-bcb3-dee007b3fa32 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:50.982010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b5e13-19d2-4e6c-acbd-c4e9ebb57895 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:50.982908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5e13-19d2-4e6c-acbd-c4e9ebb57895 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:50.983192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5e13-19d2-4e6c-acbd-c4e9ebb57895 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:50.983492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5e13-19d2-4e6c-acbd-c4e9ebb57895 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:51.100913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b5e13-19d2-4e6c-acbd-c4e9ebb57895 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:51.146057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b5e13-19d2-4e6c-acbd-c4e9ebb57895 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:51.146057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1180/2375] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:06:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:06:51.465970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1196/2376] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:06:50 2022] POST /identity/v3/users => generated 291 bytes in 708 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:06:51.483517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b5e13-19d2-4e6c-acbd-c4e9ebb57895 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:51.503660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b5e13-19d2-4e6c-acbd-c4e9ebb57895 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:51.514971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a24748-bba2-418c-ac34-f547bd059bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:51.516758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a24748-bba2-418c-ac34-f547bd059bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:51.517475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a24748-bba2-418c-ac34-f547bd059bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:51.518134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a24748-bba2-418c-ac34-f547bd059bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:51.519971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a24748-bba2-418c-ac34-f547bd059bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:51.525039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a24748-bba2-418c-ac34-f547bd059bf9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:51.542966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1181/2377] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:06:51.544954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2513e40-57c7-4238-bcb3-dee007b3fa32 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:51.560675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2513e40-57c7-4238-bcb3-dee007b3fa32 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:51.567623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1bb04-8623-4faa-84b1-50655b1572ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:51.568455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1bb04-8623-4faa-84b1-50655b1572ff admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:51.568976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1bb04-8623-4faa-84b1-50655b1572ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:51.569296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1bb04-8623-4faa-84b1-50655b1572ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:51.571150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1bb04-8623-4faa-84b1-50655b1572ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:51.573492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1bb04-8623-4faa-84b1-50655b1572ff admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:51.586754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1197/2378] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:06:51.591124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a24748-bba2-418c-ac34-f547bd059bf9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:51.619649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a24748-bba2-418c-ac34-f547bd059bf9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:51.634414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:51.638261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:51.638261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:51.638261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] PATH_INFO: `/v3/projects/e87ca5d9f50847d6ab501ca3a51ab37c/users/021817b0eef94fc2a19d90c511ec9298/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:51.677786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87ca5d9f50847d6ab501ca3a51ab37c, user_id=021817b0eef94fc2a19d90c511ec9298, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:51.686277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:51.790093 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1182/2379] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:06:51 2022] PUT /identity/v3/projects/e87ca5d9f50847d6ab501ca3a51ab37c/users/021817b0eef94fc2a19d90c511ec9298/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:06:51.796096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1afb0-bcb6-42b2-9ff8-431a46ee622f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:51.797086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1afb0-bcb6-42b2-9ff8-431a46ee622f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:51.797086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1afb0-bcb6-42b2-9ff8-431a46ee622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:51.857029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-79f1afb0-bcb6-42b2-9ff8-431a46ee622f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:51.857727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-79f1afb0-bcb6-42b2-9ff8-431a46ee622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:52.102020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1198/2380] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:52.125824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:52.128813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:52.304766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:52.344108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:52.372925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ddd81-6c00-4814-a0d2-ef2cd35f478b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:52.406213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:52.409847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:52.410435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:52.412385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:52.457379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:52.464741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:52.534698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1183/2381] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:06:53.084144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:53.771151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:54.859527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:56.545127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7751-22b6-4544-9040-39af6a34542b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:56.547132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7751-22b6-4544-9040-39af6a34542b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:56.547895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7751-22b6-4544-9040-39af6a34542b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:56.548588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:56.739935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0f0c7751-22b6-4544-9040-39af6a34542b None None] MFA Rules not processed for user `021817b0eef94fc2a19d90c511ec9298`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:06:56.748848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0f0c7751-22b6-4544-9040-39af6a34542b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:57.086620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1199/2382] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:06:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 541 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:06:57.103505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:06:57.105770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:06:57.243364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b01a56-c7ba-4185-9951-445ee2472a51 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:06:57.250427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace58a18-4df8-4a9c-9254-77e389aa66e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:06:57.251363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace58a18-4df8-4a9c-9254-77e389aa66e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:06:57.251857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace58a18-4df8-4a9c-9254-77e389aa66e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:06:57.252415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace58a18-4df8-4a9c-9254-77e389aa66e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:06:57.259778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:57.276098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace58a18-4df8-4a9c-9254-77e389aa66e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:06:57.281744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace58a18-4df8-4a9c-9254-77e389aa66e5 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:06:57.306573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1184/2383] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:06:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:06:58.080689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:59.430982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:06:59.567430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:00.934941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:01.624369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:01.830536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:02.416509 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:02.446923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:02.544811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:02.676519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:02.955763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:03.017576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:03.734808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:04.013685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:04.183340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:04.216968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:04.336916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:04.369501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:04.505974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:04.569784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:05.043800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:05.268905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:05.475741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:05.761363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:05.912837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:05.961670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:06.144330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:06.622756 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:06.738373 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:07.162099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:07.253478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:07.568710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479ac1-7e58-4b8e-b23f-a13289914ed8 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:07.569527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479ac1-7e58-4b8e-b23f-a13289914ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:07.570216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479ac1-7e58-4b8e-b23f-a13289914ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:07.606355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-73479ac1-7e58-4b8e-b23f-a13289914ed8 None None] MFA Rules not processed for user `021817b0eef94fc2a19d90c511ec9298`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:07.609370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-73479ac1-7e58-4b8e-b23f-a13289914ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:07.631915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1200/2384] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:07.772391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:07.851856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:09.130151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:09.353764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:10.493247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b77c9-b999-4e51-9364-ec59c488c7eb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:10.494188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b77c9-b999-4e51-9364-ec59c488c7eb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:10.494655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b77c9-b999-4e51-9364-ec59c488c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:10.516789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-877b77c9-b999-4e51-9364-ec59c488c7eb None None] MFA Rules not processed for user `abc2e04754884398afc999d423e2a711`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:10.522343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-877b77c9-b999-4e51-9364-ec59c488c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:10.597894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:10.688701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1185/2385] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:10.950296 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:11.608978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:12.155673 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73479ac1-7e58-4b8e-b23f-a13289914ed8 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:12.160321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-73479ac1-7e58-4b8e-b23f-a13289914ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:12.190953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73479ac1-7e58-4b8e-b23f-a13289914ed8 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:12.199001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:12.201276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:12.201791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:12.202299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] PATH_INFO: `/v3/users/e674128711f3457c9e4156c6edf24edc` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:12.204252 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e674128711f3457c9e4156c6edf24edc)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:12.207556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:12.484560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:12.518339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e674128711f3457c9e4156c6edf24edc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:12.519737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e674128711f3457c9e4156c6edf24edc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:12.520732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e674128711f3457c9e4156c6edf24edc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:12.537091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e674128711f3457c9e4156c6edf24edc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:12.549455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e674128711f3457c9e4156c6edf24edc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:12.654239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1201/2386] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:07:12 2022] DELETE /identity/v3/users/e674128711f3457c9e4156c6edf24edc => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:12.658780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a10c2-416b-49ec-a8cc-80b27c1f9c2a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:12.659273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a10c2-416b-49ec-a8cc-80b27c1f9c2a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:12.659390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a10c2-416b-49ec-a8cc-80b27c1f9c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:12.678601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e47a10c2-416b-49ec-a8cc-80b27c1f9c2a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:12.680160 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e47a10c2-416b-49ec-a8cc-80b27c1f9c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:12.783464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1186/2387] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:12.796757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:12.799392 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:12.929231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756930ec-959d-4c8a-9b1e-68c89ed850e0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:12.933707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fbd04-4ff2-4c01-9e59-644305f8f6b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:12.934269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fbd04-4ff2-4c01-9e59-644305f8f6b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:12.934553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fbd04-4ff2-4c01-9e59-644305f8f6b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:12.935160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fbd04-4ff2-4c01-9e59-644305f8f6b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:12.947272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fbd04-4ff2-4c01-9e59-644305f8f6b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:12.948857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fbd04-4ff2-4c01-9e59-644305f8f6b9 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:12.969428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1202/2388] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:07:13.661695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:13.762231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a10c2-416b-49ec-a8cc-80b27c1f9c2a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:13.766009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e47a10c2-416b-49ec-a8cc-80b27c1f9c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:13.798657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a10c2-416b-49ec-a8cc-80b27c1f9c2a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:13.810287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:13.811866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:13.812417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:13.812974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] PATH_INFO: `/v3/projects/d94f637cad9942a89fb3c98951070345` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:13.815746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d94f637cad9942a89fb3c98951070345)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:13.818388 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:13.913822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7c283-daab-40f4-9acf-a1f546a7d896 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:13.922208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7c283-daab-40f4-9acf-a1f546a7d896 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:13.922208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7c283-daab-40f4-9acf-a1f546a7d896 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:13.922208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1203/2389] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:07:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:07:13.926204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9070-76ab-4c80-a237-92fe7116caba None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:13.926485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9070-76ab-4c80-a237-92fe7116caba None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:13.926851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9070-76ab-4c80-a237-92fe7116caba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:13.985726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd94f637cad9942a89fb3c98951070345', 'request_id': 'req-3c373dc8-7ad3-4960-af1b-6059e635c801'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:13.996991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd94f637cad9942a89fb3c98951070345', 'request_id': 'req-3c373dc8-7ad3-4960-af1b-6059e635c801'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:14.012784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-00bc9070-76ab-4c80-a237-92fe7116caba None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:14.014831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-00bc9070-76ab-4c80-a237-92fe7116caba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:14.018000 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd94f637cad9942a89fb3c98951070345', 'request_id': 'req-3c373dc8-7ad3-4960-af1b-6059e635c801'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:14.133581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1204/2390] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:07:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:14.146574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc9070-76ab-4c80-a237-92fe7116caba None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:14.149028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-00bc9070-76ab-4c80-a237-92fe7116caba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:14.159570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] The token cache is being invalidate because project d94f637cad9942a89fb3c98951070345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:07:14.160028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:14.161145 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c373dc8-7ad3-4960-af1b-6059e635c801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:14.163761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1187/2391] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:07:13 2022] DELETE /identity/v3/projects/d94f637cad9942a89fb3c98951070345 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:14.172325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094866f3-e1aa-4e78-b985-f34207d2f290 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.172790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094866f3-e1aa-4e78-b985-f34207d2f290 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.174413 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094866f3-e1aa-4e78-b985-f34207d2f290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.195775 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-094866f3-e1aa-4e78-b985-f34207d2f290 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:14.197084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-094866f3-e1aa-4e78-b985-f34207d2f290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:14.267404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc9070-76ab-4c80-a237-92fe7116caba None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:14.275362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8327d9916424843b2a67cbdd856ad37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:14.276522 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.277129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.278425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.284414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:14.284678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1188/2392] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:07:14.295143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68248b-17e1-4980-b5e6-1ef4d0fa0557 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.295648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68248b-17e1-4980-b5e6-1ef4d0fa0557 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.296071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68248b-17e1-4980-b5e6-1ef4d0fa0557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.309386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:14.312127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:14.327781 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2d68248b-17e1-4980-b5e6-1ef4d0fa0557 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:14.329637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2d68248b-17e1-4980-b5e6-1ef4d0fa0557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:14.334514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1205/2393] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:07:14.347292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1189/2394] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:14.349172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:14.367028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817172-5f99-4837-bd27-21ad379d216e service glance] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:14.373132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:14.398281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957270b-05c1-49d6-ad7b-f2875183c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:14.398281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957270b-05c1-49d6-ad7b-f2875183c845 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.398281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957270b-05c1-49d6-ad7b-f2875183c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.398281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957270b-05c1-49d6-ad7b-f2875183c845 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.398586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957270b-05c1-49d6-ad7b-f2875183c845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:07:14.398586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957270b-05c1-49d6-ad7b-f2875183c845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:14.399326 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957270b-05c1-49d6-ad7b-f2875183c845 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:14.417988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1206/2395] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:07:14.422920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68248b-17e1-4980-b5e6-1ef4d0fa0557 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:14.430229 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:14.435985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68248b-17e1-4980-b5e6-1ef4d0fa0557 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:14.442081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0bd48-bc9e-4b21-b5a8-f40758f4f179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:14.442806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0bd48-bc9e-4b21-b5a8-f40758f4f179 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.443089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0bd48-bc9e-4b21-b5a8-f40758f4f179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.443398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0bd48-bc9e-4b21-b5a8-f40758f4f179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.444112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0bd48-bc9e-4b21-b5a8-f40758f4f179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:14.446153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0bd48-bc9e-4b21-b5a8-f40758f4f179 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:14.625353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1190/2396] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:07:14 2022] POST /identity/v3/projects => generated 397 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:07:14.628560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86b76-05db-49ae-b6f0-8d06bd7907d3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.628849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86b76-05db-49ae-b6f0-8d06bd7907d3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.629344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86b76-05db-49ae-b6f0-8d06bd7907d3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.631568 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1207/2397] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:07:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 04:07:14.632046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0bd48-bc9e-4b21-b5a8-f40758f4f179 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:14.638894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035f8f-9e9b-486a-873a-c359b4c2b328 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.639524 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035f8f-9e9b-486a-873a-c359b4c2b328 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.641000 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035f8f-9e9b-486a-873a-c359b4c2b328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.672631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-37035f8f-9e9b-486a-873a-c359b4c2b328 None None] MFA Rules not processed for user `0beff646206e446386d91537df9eaa67`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:14.674884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-37035f8f-9e9b-486a-873a-c359b4c2b328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:14.756973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0bd48-bc9e-4b21-b5a8-f40758f4f179 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:14.773464 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ff0f2-f06f-4eca-870c-c07cc5497f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:14.774564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1208/2398] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:14.774971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ff0f2-f06f-4eca-870c-c07cc5497f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.775516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ff0f2-f06f-4eca-870c-c07cc5497f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.776270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ff0f2-f06f-4eca-870c-c07cc5497f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.779514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ff0f2-f06f-4eca-870c-c07cc5497f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:14.785505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ff0f2-f06f-4eca-870c-c07cc5497f27 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:14.791773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37035f8f-9e9b-486a-873a-c359b4c2b328 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:14.795197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-37035f8f-9e9b-486a-873a-c359b4c2b328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:14.814836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37035f8f-9e9b-486a-873a-c359b4c2b328 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:14.823295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6926d6-59c0-49a1-b4c2-1a4ec4bc0eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:14.823295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6926d6-59c0-49a1-b4c2-1a4ec4bc0eed service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.823782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6926d6-59c0-49a1-b4c2-1a4ec4bc0eed service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.824763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6926d6-59c0-49a1-b4c2-1a4ec4bc0eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.837620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:14.842588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6926d6-59c0-49a1-b4c2-1a4ec4bc0eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:14.844021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6926d6-59c0-49a1-b4c2-1a4ec4bc0eed service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:14.854659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1209/2399] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:07:14.948668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1191/2400] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:07:14 2022] POST /identity/v3/users => generated 299 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:07:14.953745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6926d6-59c0-49a1-b4c2-1a4ec4bc0eed service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:14.968520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a921a4-9daf-434d-9fbc-da12e21cbf91 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.972958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a921a4-9daf-434d-9fbc-da12e21cbf91 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.973176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6926d6-59c0-49a1-b4c2-1a4ec4bc0eed service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:14.973323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a921a4-9daf-434d-9fbc-da12e21cbf91 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.973323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1192/2401] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:07:14.974861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a391e6c-e870-46bc-a71b-fc2a72747a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:14.975601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a391e6c-e870-46bc-a71b-fc2a72747a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.975887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a391e6c-e870-46bc-a71b-fc2a72747a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.977314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a391e6c-e870-46bc-a71b-fc2a72747a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.978041 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d684ca-ec28-4367-8406-42f85a9b230d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:14.978321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a391e6c-e870-46bc-a71b-fc2a72747a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:14.978524 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d684ca-ec28-4367-8406-42f85a9b230d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:14.978691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d684ca-ec28-4367-8406-42f85a9b230d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:14.980436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a391e6c-e870-46bc-a71b-fc2a72747a0f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:15.001832 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1210/2402] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:07:15.009758 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a391e6c-e870-46bc-a71b-fc2a72747a0f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:15.010444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-86d684ca-ec28-4367-8406-42f85a9b230d None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:15.012412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-86d684ca-ec28-4367-8406-42f85a9b230d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:15.035126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a391e6c-e870-46bc-a71b-fc2a72747a0f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:15.043714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb552b-2729-44fa-bbc3-93d23aa47872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:15.044871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb552b-2729-44fa-bbc3-93d23aa47872 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:15.045362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb552b-2729-44fa-bbc3-93d23aa47872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:15.047563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb552b-2729-44fa-bbc3-93d23aa47872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:15.047563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb552b-2729-44fa-bbc3-93d23aa47872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:15.050235 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb552b-2729-44fa-bbc3-93d23aa47872 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:15.070773 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1211/2403] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:15.076067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb552b-2729-44fa-bbc3-93d23aa47872 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:15.087934 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb552b-2729-44fa-bbc3-93d23aa47872 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:15.093390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:15.094583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:15.094868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:15.095137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] PATH_INFO: `/v3/projects/c39a004ecebb421684352e5f84ea9aa0/users/aa0ab519c9384d4f953b0caec01ddaee/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:15.113457 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c39a004ecebb421684352e5f84ea9aa0, user_id=aa0ab519c9384d4f953b0caec01ddaee, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:15.114376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1193/2404] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:07:15.116540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:15.152340 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:15.155676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d684ca-ec28-4367-8406-42f85a9b230d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:15.157487 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-86d684ca-ec28-4367-8406-42f85a9b230d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:15.174662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d684ca-ec28-4367-8406-42f85a9b230d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:15.180018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2f861-068f-4417-8264-ff635fb448d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:15.180717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2f861-068f-4417-8264-ff635fb448d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:15.181033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2f861-068f-4417-8264-ff635fb448d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:15.181289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2f861-068f-4417-8264-ff635fb448d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:15.199398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2f861-068f-4417-8264-ff635fb448d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:15.201244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2f861-068f-4417-8264-ff635fb448d7 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:15.215291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1194/2405] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:07:15.285107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1212/2406] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:07:15 2022] PUT /identity/v3/projects/c39a004ecebb421684352e5f84ea9aa0/users/aa0ab519c9384d4f953b0caec01ddaee/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:07:15.385204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438bab5-6ee8-4705-bf0c-a2d05d2a635a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:15.388169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438bab5-6ee8-4705-bf0c-a2d05d2a635a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:15.393064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438bab5-6ee8-4705-bf0c-a2d05d2a635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:15.474602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b438bab5-6ee8-4705-bf0c-a2d05d2a635a None None] MFA Rules not processed for user `aa0ab519c9384d4f953b0caec01ddaee`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:15.476012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b438bab5-6ee8-4705-bf0c-a2d05d2a635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:15.622861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1195/2407] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:15.635136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:15.637216 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:15.716939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:15.768954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efcac3-65f9-4f66-abeb-9495145c28d0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:15.773200 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:15.774019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:15.774399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:15.774735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:15.800660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:15.805091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:15.824975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1213/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:07:17.130849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:17.269458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:18.412220 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8249d2-057a-4f77-ba8d-1595a77abf39 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:18.412802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8249d2-057a-4f77-ba8d-1595a77abf39 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:18.412802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8249d2-057a-4f77-ba8d-1595a77abf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:18.422443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:18.510109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4c8249d2-057a-4f77-ba8d-1595a77abf39 None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:18.511817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4c8249d2-057a-4f77-ba8d-1595a77abf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:18.603740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1196/2409] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:07:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:07:18.616794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:18.618772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:18.733948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01325a92-2684-443d-ac37-6155bf818fd3 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:18.742529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:18.743630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:18.744131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:18.744592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:18.759414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:18.762447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:18.779038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1214/2410] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:07:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:07:19.253556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:20.018286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:20.395249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:20.881399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:21.112979 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:21.195522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:21.484665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:21.581983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:22.039022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:22.189956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:22.212699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:22.240753 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:22.412638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:22.451552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:22.629657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:22.736539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:23.088129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:23.273538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:23.315419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:23.448280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:23.709891 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:28.448238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8249d2-057a-4f77-ba8d-1595a77abf39 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:28.545906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:28.548684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:28.567916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:28.645603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8249d2-057a-4f77-ba8d-1595a77abf39 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:28.658772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:28.662490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:28.662490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:28.662490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] PATH_INFO: `/v3/users/aa0ab519c9384d4f953b0caec01ddaee` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:28.670441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0ab519c9384d4f953b0caec01ddaee)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:28.673703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:28.772523 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aebaf8-bba0-4960-b4a4-436f924928ae service swift] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:28.780085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:28.781481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:28.782063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:28.782348 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] PATH_INFO: `/v3/users/c3a66e07a127464c8c12fd6111f20369` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:28.783982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a66e07a127464c8c12fd6111f20369)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:28.786804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:28.789679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:29.069778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0ab519c9384d4f953b0caec01ddaee'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.073427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0ab519c9384d4f953b0caec01ddaee'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.171078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a66e07a127464c8c12fd6111f20369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.180037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a66e07a127464c8c12fd6111f20369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.182904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a66e07a127464c8c12fd6111f20369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.183445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0ab519c9384d4f953b0caec01ddaee'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.199818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0ab519c9384d4f953b0caec01ddaee'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.199818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6ea4543d-00cf-48e3-ab74-12827e26bad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0ab519c9384d4f953b0caec01ddaee'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.227620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a66e07a127464c8c12fd6111f20369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.249080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1197/2411] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:07:28 2022] DELETE /identity/v3/users/aa0ab519c9384d4f953b0caec01ddaee => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:29.255391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04419-fc03-433a-b9a8-cfb3a55d55ec None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:29.255709 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04419-fc03-433a-b9a8-cfb3a55d55ec None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:29.255819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6bdc9ad2-794a-4536-bf7d-4b7a230ee7e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a66e07a127464c8c12fd6111f20369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:29.255978 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04419-fc03-433a-b9a8-cfb3a55d55ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:29.298522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-22a04419-fc03-433a-b9a8-cfb3a55d55ec None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:29.301495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-22a04419-fc03-433a-b9a8-cfb3a55d55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:29.390171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1215/2412] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:07:28 2022] DELETE /identity/v3/users/c3a66e07a127464c8c12fd6111f20369 => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:07:29.401139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fae002-ad1a-4db5-99d9-8e375ee39464 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:29.401451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fae002-ad1a-4db5-99d9-8e375ee39464 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:29.401632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fae002-ad1a-4db5-99d9-8e375ee39464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:29.466670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-68fae002-ad1a-4db5-99d9-8e375ee39464 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:29.471926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-68fae002-ad1a-4db5-99d9-8e375ee39464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:29.497914 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1198/2413] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:29.516228 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a04419-fc03-433a-b9a8-cfb3a55d55ec None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:29.519752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-22a04419-fc03-433a-b9a8-cfb3a55d55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:29.678069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1216/2414] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:29.693615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fae002-ad1a-4db5-99d9-8e375ee39464 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:29.723655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a04419-fc03-433a-b9a8-cfb3a55d55ec None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:29.761640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:29.876132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:29.877820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:29.878881 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:29.879964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:29.907909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:29.912291 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:29.966878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1199/2415] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:07:29.968556 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fae002-ad1a-4db5-99d9-8e375ee39464 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:30.013938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2690cfd-6f3c-42d4-b47f-6493a8098da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:30.016820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2690cfd-6f3c-42d4-b47f-6493a8098da9 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:30.017495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2690cfd-6f3c-42d4-b47f-6493a8098da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:30.017904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2690cfd-6f3c-42d4-b47f-6493a8098da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:30.081693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2690cfd-6f3c-42d4-b47f-6493a8098da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:30.091191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2690cfd-6f3c-42d4-b47f-6493a8098da9 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:30.196868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1217/2416] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:07:30.733546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:30.737663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:30.757816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5edf0-9182-410a-8caa-0da9c3b9dd61 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:30.771561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:30.777675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:30.777675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:30.777675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] PATH_INFO: `/v3/users/a5e6a2917c5946aeb01e9619e7254c49` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:30.777675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e6a2917c5946aeb01e9619e7254c49)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:30.779345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:31.212115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e6a2917c5946aeb01e9619e7254c49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:31.216365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e6a2917c5946aeb01e9619e7254c49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:31.418348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e6a2917c5946aeb01e9619e7254c49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:31.431075 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e6a2917c5946aeb01e9619e7254c49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:31.432535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e6a2917c5946aeb01e9619e7254c49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:31.496042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1200/2417] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:07:30 2022] DELETE /identity/v3/users/a5e6a2917c5946aeb01e9619e7254c49 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:31.503162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c82c5-20a0-4727-920b-d432f10e6fb5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:31.504670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c82c5-20a0-4727-920b-d432f10e6fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:31.505266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c82c5-20a0-4727-920b-d432f10e6fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:31.542950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9b2c82c5-20a0-4727-920b-d432f10e6fb5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:31.545201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9b2c82c5-20a0-4727-920b-d432f10e6fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:31.578552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:31.683059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1218/2418] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:31.714837 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:31.716748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:31.846029 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfe67c-faac-417c-9403-e6a535d4148b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:31.850996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:31.851721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:31.852004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:31.852284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:31.868073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:31.871638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:31.884164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1201/2419] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:07:32.249686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c82c5-20a0-4727-920b-d432f10e6fb5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:32.253020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9b2c82c5-20a0-4727-920b-d432f10e6fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:32.287541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c82c5-20a0-4727-920b-d432f10e6fb5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:32.308764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:32.312214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:32.313057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:32.313882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] PATH_INFO: `/v3/projects/c39a004ecebb421684352e5f84ea9aa0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:32.316707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c39a004ecebb421684352e5f84ea9aa0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:32.320979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:32.385103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:32.387410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:32.412730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348111c-cfc9-4e4e-a5f4-09abec43ff16 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:32.419872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:32.422316 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:32.422874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:32.423389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] PATH_INFO: `/v3/projects/922d87b0a9884af798e48d5035a98e48` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:32.424988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=922d87b0a9884af798e48d5035a98e48)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:32.427168 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:32.467046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:32.499916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c39a004ecebb421684352e5f84ea9aa0', 'request_id': 'req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.507740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c39a004ecebb421684352e5f84ea9aa0', 'request_id': 'req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.592984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922d87b0a9884af798e48d5035a98e48', 'request_id': 'req-c38ad263-5ef2-4083-ad72-b1587fdefd68'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.602953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922d87b0a9884af798e48d5035a98e48', 'request_id': 'req-c38ad263-5ef2-4083-ad72-b1587fdefd68'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.612608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922d87b0a9884af798e48d5035a98e48', 'request_id': 'req-c38ad263-5ef2-4083-ad72-b1587fdefd68'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.618854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c39a004ecebb421684352e5f84ea9aa0', 'request_id': 'req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.655953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] The token cache is being invalidate because project c39a004ecebb421684352e5f84ea9aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:07:32.656334 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.657163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36cc7f90-52d7-4817-ab99-ec62f6d8e26f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.658902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1219/2420] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:07:32 2022] DELETE /identity/v3/projects/c39a004ecebb421684352e5f84ea9aa0 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:32.716841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] The token cache is being invalidate because project 922d87b0a9884af798e48d5035a98e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:07:32.717144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.718586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c38ad263-5ef2-4083-ad72-b1587fdefd68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:32.720943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1202/2421] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:07:32 2022] DELETE /identity/v3/projects/922d87b0a9884af798e48d5035a98e48 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:32.726763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caedfcc-3990-4908-a4f7-818d7b3ab269 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:32.727052 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caedfcc-3990-4908-a4f7-818d7b3ab269 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:32.727359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caedfcc-3990-4908-a4f7-818d7b3ab269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:32.780452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2caedfcc-3990-4908-a4f7-818d7b3ab269 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:32.783678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2caedfcc-3990-4908-a4f7-818d7b3ab269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:32.942398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1220/2422] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:32.951557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ed7b9-4d2f-4348-b336-4f86569f9ae3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:32.952094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ed7b9-4d2f-4348-b336-4f86569f9ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:32.952535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ed7b9-4d2f-4348-b336-4f86569f9ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:32.983931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a41ed7b9-4d2f-4348-b336-4f86569f9ae3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:32.986146 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a41ed7b9-4d2f-4348-b336-4f86569f9ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:33.000148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1203/2423] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:33.005272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caedfcc-3990-4908-a4f7-818d7b3ab269 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.034155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caedfcc-3990-4908-a4f7-818d7b3ab269 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:33.047130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:33.048386 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.048969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.049645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.051088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:07:33.051704 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:33.055017 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:33.077887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1221/2424] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:07:33.084808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ed7b9-4d2f-4348-b336-4f86569f9ae3 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.099491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ed7b9-4d2f-4348-b336-4f86569f9ae3 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:33.106650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:33.107797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.108372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.108822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.109875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:33.113198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:33.227805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1204/2425] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:07:33 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:07:33.228569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.242622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6c6bf-bda9-4625-a76e-1b3ae0f648ba admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:33.287249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:33.327152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f87f6a-29b3-4337-b45c-efc26d52cc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:33.328040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87f6a-29b3-4337-b45c-efc26d52cc31 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.328357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87f6a-29b3-4337-b45c-efc26d52cc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.328689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87f6a-29b3-4337-b45c-efc26d52cc31 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.332502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f87f6a-29b3-4337-b45c-efc26d52cc31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:33.332502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f87f6a-29b3-4337-b45c-efc26d52cc31 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:33.412438 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.414516 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:33.429651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444d073-a9db-4345-9760-a33a08673abf admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:33.436518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:33.447632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.447632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.447632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] PATH_INFO: `/v3/projects/d9797e61900c49afb98864c08dd7502e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.447632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9797e61900c49afb98864c08dd7502e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:33.447632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:33.448165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1222/2426] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:07:33 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:07:33.452364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f87f6a-29b3-4337-b45c-efc26d52cc31 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.490559 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f87f6a-29b3-4337-b45c-efc26d52cc31 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:33.504921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7fffd-9ec0-461e-b386-032dec9bacd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:33.506410 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7fffd-9ec0-461e-b386-032dec9bacd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.507141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7fffd-9ec0-461e-b386-032dec9bacd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.508066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7fffd-9ec0-461e-b386-032dec9bacd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.510461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7fffd-9ec0-461e-b386-032dec9bacd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:33.514562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7fffd-9ec0-461e-b386-032dec9bacd4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:33.537285 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1223/2427] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:33.539973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7fffd-9ec0-461e-b386-032dec9bacd4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.555885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7fffd-9ec0-461e-b386-032dec9bacd4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:33.565302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60330808-0e28-4200-93d3-f4fca6845c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:33.566465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60330808-0e28-4200-93d3-f4fca6845c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.566962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60330808-0e28-4200-93d3-f4fca6845c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.567497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60330808-0e28-4200-93d3-f4fca6845c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.568938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60330808-0e28-4200-93d3-f4fca6845c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:33.572280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60330808-0e28-4200-93d3-f4fca6845c6d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:33.599069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1224/2428] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:33.600727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60330808-0e28-4200-93d3-f4fca6845c6d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.608337 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9797e61900c49afb98864c08dd7502e', 'request_id': 'req-25e83738-a2f4-444f-b6a2-ef2dd35197f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:33.619015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60330808-0e28-4200-93d3-f4fca6845c6d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:33.621235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9797e61900c49afb98864c08dd7502e', 'request_id': 'req-25e83738-a2f4-444f-b6a2-ef2dd35197f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:33.632444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9797e61900c49afb98864c08dd7502e', 'request_id': 'req-25e83738-a2f4-444f-b6a2-ef2dd35197f9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:33.700569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9fde8-3d33-400f-b2ea-58ad351d1d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:33.701438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9fde8-3d33-400f-b2ea-58ad351d1d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.701757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9fde8-3d33-400f-b2ea-58ad351d1d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.702042 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9fde8-3d33-400f-b2ea-58ad351d1d19 admin admin] PATH_INFO: `/v3/projects/da99a87350634e9ea8a89f314f5e6118/users/2e9707561b4a4811bd7a7d3fed4fcd6b/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.720906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] The token cache is being invalidate because project d9797e61900c49afb98864c08dd7502e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:07:33.721351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:33.722068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:33.723740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1205/2429] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:07:33 2022] DELETE /identity/v3/projects/d9797e61900c49afb98864c08dd7502e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:07:33.729808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.732071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:33.732841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9fde8-3d33-400f-b2ea-58ad351d1d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da99a87350634e9ea8a89f314f5e6118, user_id=2e9707561b4a4811bd7a7d3fed4fcd6b, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:33.737584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9fde8-3d33-400f-b2ea-58ad351d1d19 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:33.750530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e83738-a2f4-444f-b6a2-ef2dd35197f9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:33.756006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:33.757003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.757299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.757570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] PATH_INFO: `/v3/users/9373bd1c34044417887ec1bf1c2ba5c9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.759003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9373bd1c34044417887ec1bf1c2ba5c9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:33.760945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:33.853450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1225/2430] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:07:33 2022] PUT /identity/v3/projects/da99a87350634e9ea8a89f314f5e6118/users/2e9707561b4a4811bd7a7d3fed4fcd6b/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:33.860091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40da224-69af-4f27-8074-0dd57ee1c8e4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:33.861536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40da224-69af-4f27-8074-0dd57ee1c8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:33.861921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40da224-69af-4f27-8074-0dd57ee1c8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:33.882108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c40da224-69af-4f27-8074-0dd57ee1c8e4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:33.883442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c40da224-69af-4f27-8074-0dd57ee1c8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:33.949961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1226/2431] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:07:33.960037 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40da224-69af-4f27-8074-0dd57ee1c8e4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:33.961422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c40da224-69af-4f27-8074-0dd57ee1c8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:33.991151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:34.020620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40da224-69af-4f27-8074-0dd57ee1c8e4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:34.024352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b790ee6-6457-4fa7-846c-cb27f662eba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:34.024998 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b790ee6-6457-4fa7-846c-cb27f662eba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:34.025277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b790ee6-6457-4fa7-846c-cb27f662eba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:34.025548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b790ee6-6457-4fa7-846c-cb27f662eba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:34.036085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b790ee6-6457-4fa7-846c-cb27f662eba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:34.038839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b790ee6-6457-4fa7-846c-cb27f662eba9 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:34.038971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9373bd1c34044417887ec1bf1c2ba5c9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:34.039600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9373bd1c34044417887ec1bf1c2ba5c9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:34.049742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1227/2432] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:07:34.120479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9373bd1c34044417887ec1bf1c2ba5c9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:34.126745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=75,keystone:DELETE=21,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:34.129696 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9373bd1c34044417887ec1bf1c2ba5c9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:34.130476 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2c3a3bb-1152-4345-a665-dfdf94ef2ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9373bd1c34044417887ec1bf1c2ba5c9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:34.150560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1206/2433] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:07:33 2022] DELETE /identity/v3/users/9373bd1c34044417887ec1bf1c2ba5c9 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:34.735564 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b790ee6-6457-4fa7-846c-cb27f662eba9 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:34.758878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b790ee6-6457-4fa7-846c-cb27f662eba9 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:34.845039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:34.846535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:34.847113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:34.847581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] PATH_INFO: `/v3/projects/46ba2c9f14ad42358a5b3aa451a709d3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:34.851163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ba2c9f14ad42358a5b3aa451a709d3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:34.855418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:35.066579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ba2c9f14ad42358a5b3aa451a709d3', 'request_id': 'req-e77c1510-8541-4362-bf25-bd86c9eb0b4f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:35.074451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ba2c9f14ad42358a5b3aa451a709d3', 'request_id': 'req-e77c1510-8541-4362-bf25-bd86c9eb0b4f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:35.202076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ba2c9f14ad42358a5b3aa451a709d3', 'request_id': 'req-e77c1510-8541-4362-bf25-bd86c9eb0b4f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:35.229505 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] The token cache is being invalidate because project 46ba2c9f14ad42358a5b3aa451a709d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:07:35.230295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:35.232408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e77c1510-8541-4362-bf25-bd86c9eb0b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:35.251757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1228/2434] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:07:34 2022] DELETE /identity/v3/projects/46ba2c9f14ad42358a5b3aa451a709d3 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:35.260961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b2e6d-3575-4823-8327-819b61ab7c53 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:35.262360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b2e6d-3575-4823-8327-819b61ab7c53 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:35.263035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b2e6d-3575-4823-8327-819b61ab7c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:35.325656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9d0b2e6d-3575-4823-8327-819b61ab7c53 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:35.328180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9d0b2e6d-3575-4823-8327-819b61ab7c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:35.428610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1207/2435] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:35.448720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3bfa-898f-4542-b755-1d57fcbbd133 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:35.449236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3bfa-898f-4542-b755-1d57fcbbd133 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:35.449592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3bfa-898f-4542-b755-1d57fcbbd133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:35.491933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-135d3bfa-898f-4542-b755-1d57fcbbd133 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:35.493295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-135d3bfa-898f-4542-b755-1d57fcbbd133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:35.504012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1229/2436] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:35.510862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b2e6d-3575-4823-8327-819b61ab7c53 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:35.526264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b2e6d-3575-4823-8327-819b61ab7c53 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:35.534232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:35.534967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:35.535268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:35.535776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:35.536689 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:07:35.537012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:35.538690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:35.554512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1208/2437] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:07:35.561384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d3bfa-898f-4542-b755-1d57fcbbd133 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:35.572970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d3bfa-898f-4542-b755-1d57fcbbd133 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:35.584428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81dfb9-d103-4cd0-b046-1e66e7b73d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:35.585798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81dfb9-d103-4cd0-b046-1e66e7b73d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:35.586422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81dfb9-d103-4cd0-b046-1e66e7b73d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:35.587177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81dfb9-d103-4cd0-b046-1e66e7b73d6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:35.593745 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81dfb9-d103-4cd0-b046-1e66e7b73d6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:35.599511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81dfb9-d103-4cd0-b046-1e66e7b73d6c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:35.789638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1230/2438] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:07:35 2022] POST /identity/v3/projects => generated 365 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:07:35.790463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:35.817548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0cc73-6367-4bd5-8775-84f2309ad989 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:35.979415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacadb96-bb6a-49f6-99c4-a475c4ef9f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:35.980337 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacadb96-bb6a-49f6-99c4-a475c4ef9f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:35.980623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacadb96-bb6a-49f6-99c4-a475c4ef9f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:35.980907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacadb96-bb6a-49f6-99c4-a475c4ef9f9c admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:35.981787 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacadb96-bb6a-49f6-99c4-a475c4ef9f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:35.984378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacadb96-bb6a-49f6-99c4-a475c4ef9f9c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:36.086728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1209/2439] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:07:35 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:07:36.093226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81dfb9-d103-4cd0-b046-1e66e7b73d6c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:36.107684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81dfb9-d103-4cd0-b046-1e66e7b73d6c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:36.113662 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c85a6-9cb7-4f6e-9a85-34b7605a5285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:36.114557 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c85a6-9cb7-4f6e-9a85-34b7605a5285 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:36.115016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c85a6-9cb7-4f6e-9a85-34b7605a5285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:36.115534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c85a6-9cb7-4f6e-9a85-34b7605a5285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:36.116474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c85a6-9cb7-4f6e-9a85-34b7605a5285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:36.118994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c85a6-9cb7-4f6e-9a85-34b7605a5285 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:36.133035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1231/2440] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:36.143928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacadb96-bb6a-49f6-99c4-a475c4ef9f9c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:36.155553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacadb96-bb6a-49f6-99c4-a475c4ef9f9c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:36.161962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ff2d-12de-40f6-8af4-d76f4e201992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:36.162806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ff2d-12de-40f6-8af4-d76f4e201992 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:36.163172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ff2d-12de-40f6-8af4-d76f4e201992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:36.163481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ff2d-12de-40f6-8af4-d76f4e201992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:36.164532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ff2d-12de-40f6-8af4-d76f4e201992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:36.166323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ff2d-12de-40f6-8af4-d76f4e201992 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:36.179688 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1210/2441] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:36.184360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c85a6-9cb7-4f6e-9a85-34b7605a5285 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:36.195723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c85a6-9cb7-4f6e-9a85-34b7605a5285 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:36.201520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5450-1768-4f4a-a673-6ffac10264e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:36.202453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5450-1768-4f4a-a673-6ffac10264e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:36.202766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5450-1768-4f4a-a673-6ffac10264e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:36.203015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5450-1768-4f4a-a673-6ffac10264e4 admin admin] PATH_INFO: `/v3/projects/aebde2e4d33144d793dd5b6e3adb4e8e/users/3d32f9cbbc404dcf8ea51117c223faab/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:36.222379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf5450-1768-4f4a-a673-6ffac10264e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aebde2e4d33144d793dd5b6e3adb4e8e, user_id=3d32f9cbbc404dcf8ea51117c223faab, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:36.225158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf5450-1768-4f4a-a673-6ffac10264e4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:36.227936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:36.308576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1232/2442] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:07:36 2022] PUT /identity/v3/projects/aebde2e4d33144d793dd5b6e3adb4e8e/users/3d32f9cbbc404dcf8ea51117c223faab/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:07:36.326878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e534-1d0a-4020-a092-e52ec70eed7f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:36.328251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e534-1d0a-4020-a092-e52ec70eed7f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:36.328886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e534-1d0a-4020-a092-e52ec70eed7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:36.377376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-73f2e534-1d0a-4020-a092-e52ec70eed7f None None] MFA Rules not processed for user `3d32f9cbbc404dcf8ea51117c223faab`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:36.378766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-73f2e534-1d0a-4020-a092-e52ec70eed7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:36.441598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:36.502750 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d95787-b349-4e24-bcc9-990549ae83db None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:36.503075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d95787-b349-4e24-bcc9-990549ae83db None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:36.503356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d95787-b349-4e24-bcc9-990549ae83db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:36.504165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1211/2443] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:36.516474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2e534-1d0a-4020-a092-e52ec70eed7f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:36.539549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d1d95787-b349-4e24-bcc9-990549ae83db None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:36.540946 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d1d95787-b349-4e24-bcc9-990549ae83db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:36.650063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2e534-1d0a-4020-a092-e52ec70eed7f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:36.667548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1233/2444] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:07:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:07:36.668471 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8e57f-61ab-4e38-aba1-9b496ee6a1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:36.669569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8e57f-61ab-4e38-aba1-9b496ee6a1fc admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:36.669569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8e57f-61ab-4e38-aba1-9b496ee6a1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:36.669812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8e57f-61ab-4e38-aba1-9b496ee6a1fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:36.670500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8e57f-61ab-4e38-aba1-9b496ee6a1fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:36.673022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8e57f-61ab-4e38-aba1-9b496ee6a1fc admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:36.699545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d95787-b349-4e24-bcc9-990549ae83db None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:36.701064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d1d95787-b349-4e24-bcc9-990549ae83db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:36.772351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1212/2445] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:07:36 2022] POST /identity/v3/projects => generated 363 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:07:36.773508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8e57f-61ab-4e38-aba1-9b496ee6a1fc admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:36.839899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d95787-b349-4e24-bcc9-990549ae83db None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:36.854378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8e57f-61ab-4e38-aba1-9b496ee6a1fc admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:36.859982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b9474-1e48-4165-8c09-07e8a1dddc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:36.860418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9474-1e48-4165-8c09-07e8a1dddc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:36.860707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9474-1e48-4165-8c09-07e8a1dddc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:36.861019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9474-1e48-4165-8c09-07e8a1dddc62 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:36.861725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b9474-1e48-4165-8c09-07e8a1dddc62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:36.863740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b9474-1e48-4165-8c09-07e8a1dddc62 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:36.985208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242444e-6043-4bdd-9e35-b3583545e9d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:36.985861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242444e-6043-4bdd-9e35-b3583545e9d3 service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:36.986185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242444e-6043-4bdd-9e35-b3583545e9d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:36.986474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242444e-6043-4bdd-9e35-b3583545e9d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:37.033206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1213/2446] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:07:36 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:07:37.036600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b9474-1e48-4165-8c09-07e8a1dddc62 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:37.049808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b9474-1e48-4165-8c09-07e8a1dddc62 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:37.055525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:37.072295 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6725d-69f7-440d-8682-d382afc168d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:37.073802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6725d-69f7-440d-8682-d382afc168d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:37.074301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6725d-69f7-440d-8682-d382afc168d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:37.074733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6725d-69f7-440d-8682-d382afc168d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:37.075786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6725d-69f7-440d-8682-d382afc168d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:37.077694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6725d-69f7-440d-8682-d382afc168d3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:37.078747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242444e-6043-4bdd-9e35-b3583545e9d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:37.080917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242444e-6043-4bdd-9e35-b3583545e9d3 service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:37.093046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1214/2447] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:37.100904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1234/2448] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:07:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:07:37.101243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6725d-69f7-440d-8682-d382afc168d3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:37.127009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6725d-69f7-440d-8682-d382afc168d3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:37.142304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d6cba-3ecf-4167-a123-5f7407e3bdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:37.143885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d6cba-3ecf-4167-a123-5f7407e3bdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:37.146292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d6cba-3ecf-4167-a123-5f7407e3bdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:37.146715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d6cba-3ecf-4167-a123-5f7407e3bdbd admin admin] PATH_INFO: `/v3/projects/3f70a0af77bc46c391c8c467fe9262ff/users/c3302923caac487ca51c633c3dd6bba6/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:37.197580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d6cba-3ecf-4167-a123-5f7407e3bdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f70a0af77bc46c391c8c467fe9262ff, user_id=c3302923caac487ca51c633c3dd6bba6, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:37.201248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d6cba-3ecf-4167-a123-5f7407e3bdbd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:37.320984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1215/2449] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:07:37 2022] PUT /identity/v3/projects/3f70a0af77bc46c391c8c467fe9262ff/users/c3302923caac487ca51c633c3dd6bba6/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:37.324859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242444e-6043-4bdd-9e35-b3583545e9d3 service swift] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:37.377912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242444e-6043-4bdd-9e35-b3583545e9d3 service swift] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:37.646172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc39cf6-dd94-4e03-803e-9f06734bd77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:37.647808 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39cf6-dd94-4e03-803e-9f06734bd77a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:37.648244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39cf6-dd94-4e03-803e-9f06734bd77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:37.648705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39cf6-dd94-4e03-803e-9f06734bd77a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:37.654137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc39cf6-dd94-4e03-803e-9f06734bd77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:37.654137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc39cf6-dd94-4e03-803e-9f06734bd77a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:37.728775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1235/2450] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:37.730710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d6cba-3ecf-4167-a123-5f7407e3bdbd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:37.758283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d6cba-3ecf-4167-a123-5f7407e3bdbd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:37.784071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:37.787607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:37.788107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:37.791025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] PATH_INFO: `/v3/projects/3f70a0af77bc46c391c8c467fe9262ff/users/c3302923caac487ca51c633c3dd6bba6/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:37.800399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f70a0af77bc46c391c8c467fe9262ff, user_id=c3302923caac487ca51c633c3dd6bba6, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:37.810418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:37.837630 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:37.848847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1216/2451] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:07:37 2022] PUT /identity/v3/projects/3f70a0af77bc46c391c8c467fe9262ff/users/c3302923caac487ca51c633c3dd6bba6/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:07:37.865142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d00e40-240c-4961-b4ca-7bc90cc4b06b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:37.865604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d00e40-240c-4961-b4ca-7bc90cc4b06b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:37.868771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d00e40-240c-4961-b4ca-7bc90cc4b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:37.919202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-37d00e40-240c-4961-b4ca-7bc90cc4b06b None None] MFA Rules not processed for user `c3302923caac487ca51c633c3dd6bba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:37.920954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-37d00e40-240c-4961-b4ca-7bc90cc4b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:38.044461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1236/2452] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:37 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:38.061313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:38.066099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:38.302861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f10f4-8c47-4ed8-9fd8-ac2579e4e935 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:38.317167 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57469f1f-e22e-4eb0-b591-7f74e591bf8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:38.318622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57469f1f-e22e-4eb0-b591-7f74e591bf8e service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:38.319481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57469f1f-e22e-4eb0-b591-7f74e591bf8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:38.320172 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57469f1f-e22e-4eb0-b591-7f74e591bf8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:38.357214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57469f1f-e22e-4eb0-b591-7f74e591bf8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:38.360012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57469f1f-e22e-4eb0-b591-7f74e591bf8e service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:38.385921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1217/2453] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:07:39.359374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d00e40-240c-4961-b4ca-7bc90cc4b06b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:39.380253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d00e40-240c-4961-b4ca-7bc90cc4b06b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:39.380253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:39.380253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:39.380253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:39.380709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:39.555671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:39.558109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:39.571657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1237/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:07:48.378722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=26 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:49.161936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de809c-4e70-47b1-a7a5-26bab95b2d1f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:49.162998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de809c-4e70-47b1-a7a5-26bab95b2d1f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:49.167261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de809c-4e70-47b1-a7a5-26bab95b2d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:49.290986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-63de809c-4e70-47b1-a7a5-26bab95b2d1f None None] MFA Rules not processed for user `2e9707561b4a4811bd7a7d3fed4fcd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:49.294843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-63de809c-4e70-47b1-a7a5-26bab95b2d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:49.493911 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1218/2455] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:49.495660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:49.497384 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:49.520115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:49.922507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394b817-2599-48bc-bf25-ef27a4fd6e9b service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:49.941516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fdac3-2fb0-4866-b9b2-e4229da4a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:49.943945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fdac3-2fb0-4866-b9b2-e4229da4a1ca admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:49.944778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fdac3-2fb0-4866-b9b2-e4229da4a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:49.957670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fdac3-2fb0-4866-b9b2-e4229da4a1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:49.976807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fdac3-2fb0-4866-b9b2-e4229da4a1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:49.999261 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fdac3-2fb0-4866-b9b2-e4229da4a1ca admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:50.148875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1238/2456] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:07:49 2022] POST /identity/v3/projects => generated 377 bytes in 654 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:07:50.167591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de809c-4e70-47b1-a7a5-26bab95b2d1f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:50.185062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de809c-4e70-47b1-a7a5-26bab95b2d1f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:50.346841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bdaab-459a-4d4b-ac5e-f307552c0f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:50.347607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bdaab-459a-4d4b-ac5e-f307552c0f82 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:50.347889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bdaab-459a-4d4b-ac5e-f307552c0f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:50.348211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bdaab-459a-4d4b-ac5e-f307552c0f82 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:50.348876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bdaab-459a-4d4b-ac5e-f307552c0f82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:50.350966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bdaab-459a-4d4b-ac5e-f307552c0f82 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:50.814387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1219/2457] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:07:50 2022] POST /identity/v3/users => generated 295 bytes in 653 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:07:50.816153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fdac3-2fb0-4866-b9b2-e4229da4a1ca admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:50.828273 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fdac3-2fb0-4866-b9b2-e4229da4a1ca admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:50.838680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4d222-0c2f-448f-a7ef-35f9b5d449a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:50.839981 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4d222-0c2f-448f-a7ef-35f9b5d449a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:50.840380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4d222-0c2f-448f-a7ef-35f9b5d449a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:50.840731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4d222-0c2f-448f-a7ef-35f9b5d449a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:50.841633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4d222-0c2f-448f-a7ef-35f9b5d449a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:50.843613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4d222-0c2f-448f-a7ef-35f9b5d449a2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:50.859267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1239/2458] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:50.863884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bdaab-459a-4d4b-ac5e-f307552c0f82 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:50.880495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bdaab-459a-4d4b-ac5e-f307552c0f82 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:50.889469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b130e8-a1b3-4bb4-949e-30f01346f2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:50.889469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b130e8-a1b3-4bb4-949e-30f01346f2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:50.889469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b130e8-a1b3-4bb4-949e-30f01346f2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:50.889469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b130e8-a1b3-4bb4-949e-30f01346f2b7 admin admin] PATH_INFO: `/v3/projects/2977fbb8fa004c1ba5f9572e2b043c28/users/4966505aa6de4ef1839eeab044dc928f/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:50.926289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b130e8-a1b3-4bb4-949e-30f01346f2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2977fbb8fa004c1ba5f9572e2b043c28, user_id=4966505aa6de4ef1839eeab044dc928f, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:50.933714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b130e8-a1b3-4bb4-949e-30f01346f2b7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:51.065845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1220/2459] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:07:50 2022] PUT /identity/v3/projects/2977fbb8fa004c1ba5f9572e2b043c28/users/4966505aa6de4ef1839eeab044dc928f/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:07:51.072375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4d222-0c2f-448f-a7ef-35f9b5d449a2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:51.101912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4d222-0c2f-448f-a7ef-35f9b5d449a2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:51.241795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cce9a-b505-43bc-a3aa-45d2fdd30fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:51.242971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cce9a-b505-43bc-a3aa-45d2fdd30fed admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:51.243312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cce9a-b505-43bc-a3aa-45d2fdd30fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:51.243594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cce9a-b505-43bc-a3aa-45d2fdd30fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:51.245772 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cce9a-b505-43bc-a3aa-45d2fdd30fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:51.247531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cce9a-b505-43bc-a3aa-45d2fdd30fed admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:51.276931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1240/2460] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:07:51.280826 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b130e8-a1b3-4bb4-949e-30f01346f2b7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:51.315839 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b130e8-a1b3-4bb4-949e-30f01346f2b7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:51.326838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f84b0-c27a-406a-bcde-dbb89163e993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:51.330330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f84b0-c27a-406a-bcde-dbb89163e993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:51.330958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f84b0-c27a-406a-bcde-dbb89163e993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:51.331561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f84b0-c27a-406a-bcde-dbb89163e993 admin admin] PATH_INFO: `/v3/projects/2977fbb8fa004c1ba5f9572e2b043c28/users/4966505aa6de4ef1839eeab044dc928f/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:51.358204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f84b0-c27a-406a-bcde-dbb89163e993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2977fbb8fa004c1ba5f9572e2b043c28, user_id=4966505aa6de4ef1839eeab044dc928f, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:51.384984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f84b0-c27a-406a-bcde-dbb89163e993 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:51.413827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1221/2461] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:07:51 2022] PUT /identity/v3/projects/2977fbb8fa004c1ba5f9572e2b043c28/users/4966505aa6de4ef1839eeab044dc928f/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:07:57.030336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cce9a-b505-43bc-a3aa-45d2fdd30fed admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:57.048360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cce9a-b505-43bc-a3aa-45d2fdd30fed admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:57.147535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:57.148665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:57.149049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:57.149420 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] PATH_INFO: `/v3/users/3d32f9cbbc404dcf8ea51117c223faab` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:57.151070 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d32f9cbbc404dcf8ea51117c223faab)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:57.153515 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:57.175725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:57.472559 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d32f9cbbc404dcf8ea51117c223faab'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:57.474161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d32f9cbbc404dcf8ea51117c223faab'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:57.474161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d32f9cbbc404dcf8ea51117c223faab'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:57.489544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d32f9cbbc404dcf8ea51117c223faab'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:57.501279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d32f9cbbc404dcf8ea51117c223faab'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:57.619212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1241/2462] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:07:57 2022] DELETE /identity/v3/users/3d32f9cbbc404dcf8ea51117c223faab => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:57.623953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de90c3-791a-48ea-b526-6c464c7bcdd8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:57.623953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de90c3-791a-48ea-b526-6c464c7bcdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:57.623953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de90c3-791a-48ea-b526-6c464c7bcdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:57.657649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c8de90c3-791a-48ea-b526-6c464c7bcdd8 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:57.659837 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c8de90c3-791a-48ea-b526-6c464c7bcdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:57.750811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:57.829511 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1222/2463] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:57.838748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:57.839471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:57.991100 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bdfe5-1dd1-4a68-918f-418e5208c334 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:57.997569 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826df148-abbc-4010-a932-7704debbf60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:58.000715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df148-abbc-4010-a932-7704debbf60d service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:58.000715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df148-abbc-4010-a932-7704debbf60d service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:58.000715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df148-abbc-4010-a932-7704debbf60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:58.021626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826df148-abbc-4010-a932-7704debbf60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:58.023283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826df148-abbc-4010-a932-7704debbf60d service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:58.039305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1242/2464] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:07:58.561835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de90c3-791a-48ea-b526-6c464c7bcdd8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:58.563482 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c8de90c3-791a-48ea-b526-6c464c7bcdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:58.582690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de90c3-791a-48ea-b526-6c464c7bcdd8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:58.588076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:58.589163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:58.589695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:58.590127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] PATH_INFO: `/v3/projects/aebde2e4d33144d793dd5b6e3adb4e8e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:58.591648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aebde2e4d33144d793dd5b6e3adb4e8e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:58.594068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:58.778663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aebde2e4d33144d793dd5b6e3adb4e8e', 'request_id': 'req-7dc7d127-2e8d-4723-a01c-a5a52fc72354'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:58.791967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aebde2e4d33144d793dd5b6e3adb4e8e', 'request_id': 'req-7dc7d127-2e8d-4723-a01c-a5a52fc72354'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:58.800506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aebde2e4d33144d793dd5b6e3adb4e8e', 'request_id': 'req-7dc7d127-2e8d-4723-a01c-a5a52fc72354'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:58.950879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] The token cache is being invalidate because project aebde2e4d33144d793dd5b6e3adb4e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:07:58.951175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:58.978327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7dc7d127-2e8d-4723-a01c-a5a52fc72354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:58.987085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1223/2465] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:07:58 2022] DELETE /identity/v3/projects/aebde2e4d33144d793dd5b6e3adb4e8e => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:58.996057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826df148-abbc-4010-a932-7704debbf60d service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:59.009767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826df148-abbc-4010-a932-7704debbf60d service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:59.105307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:59.107547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:59.108203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:59.108660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] PATH_INFO: `/v3/users/c3302923caac487ca51c633c3dd6bba6` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:59.110407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3302923caac487ca51c633c3dd6bba6)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:59.112943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:59.177574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b9f5e-b7fb-4b90-8420-5e679eca2f8c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:59.178038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b9f5e-b7fb-4b90-8420-5e679eca2f8c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:59.178447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b9f5e-b7fb-4b90-8420-5e679eca2f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:59.229001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c44b9f5e-b7fb-4b90-8420-5e679eca2f8c None None] MFA Rules not processed for user `4966505aa6de4ef1839eeab044dc928f`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:07:59.231168 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c44b9f5e-b7fb-4b90-8420-5e679eca2f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:59.304151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:59.321367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3302923caac487ca51c633c3dd6bba6'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:59.322372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3302923caac487ca51c633c3dd6bba6'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:59.323923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3302923caac487ca51c633c3dd6bba6'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:59.337636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1224/2466] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:07:59.348371 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3302923caac487ca51c633c3dd6bba6'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:59.359916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3302923caac487ca51c633c3dd6bba6'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:07:59.360052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b9f5e-b7fb-4b90-8420-5e679eca2f8c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:07:59.360538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c44b9f5e-b7fb-4b90-8420-5e679eca2f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:59.441755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:07:59.443973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1243/2467] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:07:58 2022] DELETE /identity/v3/users/c3302923caac487ca51c633c3dd6bba6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:07:59.503701 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b9f5e-b7fb-4b90-8420-5e679eca2f8c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:07:59.652246 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e8116-3179-4db8-b3ef-08dfcbc01bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:07:59.657628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e8116-3179-4db8-b3ef-08dfcbc01bd2 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:07:59.658050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e8116-3179-4db8-b3ef-08dfcbc01bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:07:59.658343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e8116-3179-4db8-b3ef-08dfcbc01bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:07:59.663889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-da7e8116-3179-4db8-b3ef-08dfcbc01bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:07:59.763830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e8116-3179-4db8-b3ef-08dfcbc01bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:07:59.763830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e8116-3179-4db8-b3ef-08dfcbc01bd2 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:07:59.771517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1225/2468] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:07:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:08:00.218212 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:00.222317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:00.363302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59aab-4fdb-483c-bdb7-d0a321aefbc6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:00.369328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:00.370246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:00.370536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:00.370817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] PATH_INFO: `/v3/projects/3f70a0af77bc46c391c8c467fe9262ff` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:00.373188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f70a0af77bc46c391c8c467fe9262ff)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:00.374292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:00.577208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f70a0af77bc46c391c8c467fe9262ff', 'request_id': 'req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:00.590636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f70a0af77bc46c391c8c467fe9262ff', 'request_id': 'req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:00.701950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f70a0af77bc46c391c8c467fe9262ff', 'request_id': 'req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:00.726570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] The token cache is being invalidate because project 3f70a0af77bc46c391c8c467fe9262ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:08:00.726860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:00.727551 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-541efcdf-b76a-4ac7-b86d-bdf9c9343b29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:00.729225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1244/2469] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:08:00 2022] DELETE /identity/v3/projects/3f70a0af77bc46c391c8c467fe9262ff => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:00.741819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0ce3-c7e8-4573-b792-5fff0b273459 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:00.742402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0ce3-c7e8-4573-b792-5fff0b273459 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:00.742767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0ce3-c7e8-4573-b792-5fff0b273459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:00.770192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-129d0ce3-c7e8-4573-b792-5fff0b273459 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:00.771598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-129d0ce3-c7e8-4573-b792-5fff0b273459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:00.860142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1226/2470] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:00.870967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca8054-edbb-4bc9-9f98-5e1b72ec5b2d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:00.871484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca8054-edbb-4bc9-9f98-5e1b72ec5b2d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:00.871883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca8054-edbb-4bc9-9f98-5e1b72ec5b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:00.904653 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d4ca8054-edbb-4bc9-9f98-5e1b72ec5b2d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:00.906650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d4ca8054-edbb-4bc9-9f98-5e1b72ec5b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:00.918717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1245/2471] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:00.923554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d0ce3-c7e8-4573-b792-5fff0b273459 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:00.935372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d0ce3-c7e8-4573-b792-5fff0b273459 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:00.941007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:00.941705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:00.941992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:00.947515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:00.947515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:08:00.947515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:00.947515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:00.961116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1227/2472] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 04:08:00.965840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca8054-edbb-4bc9-9f98-5e1b72ec5b2d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:00.978826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca8054-edbb-4bc9-9f98-5e1b72ec5b2d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:00.984461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ccd25-1b3f-4760-966a-e1410286a75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:00.985398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ccd25-1b3f-4760-966a-e1410286a75b admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:00.986160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ccd25-1b3f-4760-966a-e1410286a75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:00.986450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ccd25-1b3f-4760-966a-e1410286a75b admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:00.987266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ccd25-1b3f-4760-966a-e1410286a75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:00.991041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ccd25-1b3f-4760-966a-e1410286a75b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:01.121071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1246/2473] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:08:00 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:08:01.125311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:01.139009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea59a22-1a89-4bae-af98-3bee46b079ca admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:01.167921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:01.257700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc14610-2c22-4d52-9db2-6459eaa4f5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:01.258401 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14610-2c22-4d52-9db2-6459eaa4f5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:01.258718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14610-2c22-4d52-9db2-6459eaa4f5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:01.258994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14610-2c22-4d52-9db2-6459eaa4f5ec admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:01.259757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc14610-2c22-4d52-9db2-6459eaa4f5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:01.261954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc14610-2c22-4d52-9db2-6459eaa4f5ec admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:01.400853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1228/2474] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:08:01 2022] POST /identity/v3/users => generated 288 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:08:01.407351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ccd25-1b3f-4760-966a-e1410286a75b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:01.426252 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ccd25-1b3f-4760-966a-e1410286a75b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:01.435383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731b7e7-2d42-4a75-aa91-59c0dc8b4d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:01.437311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731b7e7-2d42-4a75-aa91-59c0dc8b4d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:01.437835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731b7e7-2d42-4a75-aa91-59c0dc8b4d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:01.438245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731b7e7-2d42-4a75-aa91-59c0dc8b4d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:01.439631 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731b7e7-2d42-4a75-aa91-59c0dc8b4d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:01.441570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731b7e7-2d42-4a75-aa91-59c0dc8b4d8f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:01.461671 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1247/2475] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:08:01.464473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc14610-2c22-4d52-9db2-6459eaa4f5ec admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:01.481031 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc14610-2c22-4d52-9db2-6459eaa4f5ec admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:01.486760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db12c3-e90c-42eb-a787-268d670bb609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:01.487715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db12c3-e90c-42eb-a787-268d670bb609 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:01.488717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db12c3-e90c-42eb-a787-268d670bb609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:01.488717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db12c3-e90c-42eb-a787-268d670bb609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:01.489467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db12c3-e90c-42eb-a787-268d670bb609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:01.492103 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db12c3-e90c-42eb-a787-268d670bb609 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:01.507299 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1229/2476] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:08:01.511036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731b7e7-2d42-4a75-aa91-59c0dc8b4d8f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:01.525098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:01.533622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731b7e7-2d42-4a75-aa91-59c0dc8b4d8f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:01.543685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54de19a-6206-43ca-be84-96c489350ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:01.545423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54de19a-6206-43ca-be84-96c489350ab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:01.546139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54de19a-6206-43ca-be84-96c489350ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:01.546597 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54de19a-6206-43ca-be84-96c489350ab4 admin admin] PATH_INFO: `/v3/projects/a4b6d1b30915472c876836b1500b3521/users/4e521f459d8c4a3dbad8c6cee7f1fa42/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:01.572184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54de19a-6206-43ca-be84-96c489350ab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b6d1b30915472c876836b1500b3521, user_id=4e521f459d8c4a3dbad8c6cee7f1fa42, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:01.576936 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54de19a-6206-43ca-be84-96c489350ab4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:01.741224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1248/2477] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:08:01 2022] PUT /identity/v3/projects/a4b6d1b30915472c876836b1500b3521/users/4e521f459d8c4a3dbad8c6cee7f1fa42/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:01.774717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18d7c-4cfa-499a-ba1b-3f2072ee7765 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:01.774717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18d7c-4cfa-499a-ba1b-3f2072ee7765 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:01.774717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18d7c-4cfa-499a-ba1b-3f2072ee7765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:01.881965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f3a18d7c-4cfa-499a-ba1b-3f2072ee7765 None None] MFA Rules not processed for user `4e521f459d8c4a3dbad8c6cee7f1fa42`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:01.890915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f3a18d7c-4cfa-499a-ba1b-3f2072ee7765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:02.098910 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1230/2478] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:02.103850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54de19a-6206-43ca-be84-96c489350ab4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:02.233788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54de19a-6206-43ca-be84-96c489350ab4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:02.238908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b19e40-6de0-4a92-a53e-097a9a2721f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:02.239657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b19e40-6de0-4a92-a53e-097a9a2721f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:02.239966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b19e40-6de0-4a92-a53e-097a9a2721f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:02.240245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b19e40-6de0-4a92-a53e-097a9a2721f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:02.241002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b19e40-6de0-4a92-a53e-097a9a2721f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:02.242953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b19e40-6de0-4a92-a53e-097a9a2721f5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:02.389350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1249/2479] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:08:02 2022] POST /identity/v3/projects => generated 377 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:08:02.394060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a18d7c-4cfa-499a-ba1b-3f2072ee7765 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:02.445449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a18d7c-4cfa-499a-ba1b-3f2072ee7765 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:02.531887 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfd6e-5f02-41d4-b89e-6183d370e84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:02.533208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfd6e-5f02-41d4-b89e-6183d370e84d admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:02.533208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfd6e-5f02-41d4-b89e-6183d370e84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:02.533507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfd6e-5f02-41d4-b89e-6183d370e84d admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:02.533990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cfd6e-5f02-41d4-b89e-6183d370e84d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:02.536066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cfd6e-5f02-41d4-b89e-6183d370e84d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:02.648612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1231/2480] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:08:02 2022] POST /identity/v3/users => generated 295 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:08:02.650245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b19e40-6de0-4a92-a53e-097a9a2721f5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:02.664954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b19e40-6de0-4a92-a53e-097a9a2721f5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:02.671096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947fc66-4e6f-4de1-824a-f34e2d3ef6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:02.671831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947fc66-4e6f-4de1-824a-f34e2d3ef6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:02.672103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947fc66-4e6f-4de1-824a-f34e2d3ef6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:02.672579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947fc66-4e6f-4de1-824a-f34e2d3ef6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:02.673532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947fc66-4e6f-4de1-824a-f34e2d3ef6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:02.675173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947fc66-4e6f-4de1-824a-f34e2d3ef6ec admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:02.687983 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1250/2481] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:08:02.696811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfd6e-5f02-41d4-b89e-6183d370e84d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:02.718446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfd6e-5f02-41d4-b89e-6183d370e84d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:02.724127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae0d9-26e1-45cd-8da8-80588862dabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:02.725028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae0d9-26e1-45cd-8da8-80588862dabd admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:02.725307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae0d9-26e1-45cd-8da8-80588862dabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:02.725586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae0d9-26e1-45cd-8da8-80588862dabd admin admin] PATH_INFO: `/v3/projects/fc75a91eab9940468f8b57dd4a871f46/users/cae23d8b355b47f5b9a1e755a5c48250/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:02.736710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:02.743664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ae0d9-26e1-45cd-8da8-80588862dabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc75a91eab9940468f8b57dd4a871f46, user_id=cae23d8b355b47f5b9a1e755a5c48250, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:02.746500 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ae0d9-26e1-45cd-8da8-80588862dabd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:02.883396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1232/2482] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:08:02 2022] PUT /identity/v3/projects/fc75a91eab9940468f8b57dd4a871f46/users/cae23d8b355b47f5b9a1e755a5c48250/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:02.890080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947fc66-4e6f-4de1-824a-f34e2d3ef6ec admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:02.923030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947fc66-4e6f-4de1-824a-f34e2d3ef6ec admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:03.132652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45709859-be61-4434-af66-3a62b96e80e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:03.133489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45709859-be61-4434-af66-3a62b96e80e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:03.133780 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45709859-be61-4434-af66-3a62b96e80e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:03.134061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45709859-be61-4434-af66-3a62b96e80e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:03.134872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45709859-be61-4434-af66-3a62b96e80e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:03.140657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45709859-be61-4434-af66-3a62b96e80e6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:03.177418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1251/2483] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:08:03.184463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae0d9-26e1-45cd-8da8-80588862dabd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:03.203954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae0d9-26e1-45cd-8da8-80588862dabd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:03.235058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:03.236711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:03.237251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:03.238117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] PATH_INFO: `/v3/projects/fc75a91eab9940468f8b57dd4a871f46/users/cae23d8b355b47f5b9a1e755a5c48250/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:03.246082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc75a91eab9940468f8b57dd4a871f46, user_id=cae23d8b355b47f5b9a1e755a5c48250, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:03.252210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:03.291847 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1233/2484] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:08:03 2022] PUT /identity/v3/projects/fc75a91eab9940468f8b57dd4a871f46/users/cae23d8b355b47f5b9a1e755a5c48250/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:03.295610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a932a-ebd5-4db3-a7f0-b1b12515dd03 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:03.295909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a932a-ebd5-4db3-a7f0-b1b12515dd03 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:03.296180 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a932a-ebd5-4db3-a7f0-b1b12515dd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:03.386411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f77a932a-ebd5-4db3-a7f0-b1b12515dd03 None None] MFA Rules not processed for user `cae23d8b355b47f5b9a1e755a5c48250`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:03.393015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f77a932a-ebd5-4db3-a7f0-b1b12515dd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:03.526899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:03.612717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:03.615381 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:03.617271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1252/2485] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:03.636059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a932a-ebd5-4db3-a7f0-b1b12515dd03 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:03.638869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f77a932a-ebd5-4db3-a7f0-b1b12515dd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:03.752560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a932a-ebd5-4db3-a7f0-b1b12515dd03 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:03.753236 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba682d45-6d2b-4d54-b21e-55bb368ecede admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:03.757830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976a9b6-310b-49fb-bc54-bd90fb6ad737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:03.758819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976a9b6-310b-49fb-bc54-bd90fb6ad737 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:03.759292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976a9b6-310b-49fb-bc54-bd90fb6ad737 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:03.760056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976a9b6-310b-49fb-bc54-bd90fb6ad737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:03.767366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:03.768054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:03.768559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:03.769095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:03.774162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:03.856588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976a9b6-310b-49fb-bc54-bd90fb6ad737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:03.858055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976a9b6-310b-49fb-bc54-bd90fb6ad737 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:03.872458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1253/2486] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:08:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:08:03.907546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:03.909676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:03.948604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1234/2487] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:08:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:08:04.466563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ecc4c-620f-49c1-b600-410c26e863a3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:04.467344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ecc4c-620f-49c1-b600-410c26e863a3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:04.467921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ecc4c-620f-49c1-b600-410c26e863a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:04.501706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-602ecc4c-620f-49c1-b600-410c26e863a3 None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:04.505964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-602ecc4c-620f-49c1-b600-410c26e863a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:04.615430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1254/2488] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 1 04:08:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:04.632403 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:04.634065 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:04.788249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5871661-3eec-4712-9fb5-94bc7d84ab50 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:04.802412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:04.803100 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:04.803450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:04.803791 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:04.834469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:04.837625 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:04.850173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1235/2489] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:08:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 04:08:10.900111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ecc4c-620f-49c1-b600-410c26e863a3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:10.912502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ecc4c-620f-49c1-b600-410c26e863a3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:10.916529 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636e711-8c31-4478-a98d-5f3e5413aee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:10.917466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636e711-8c31-4478-a98d-5f3e5413aee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:10.917846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636e711-8c31-4478-a98d-5f3e5413aee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:10.918281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636e711-8c31-4478-a98d-5f3e5413aee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:10.929061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636e711-8c31-4478-a98d-5f3e5413aee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:10.930599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636e711-8c31-4478-a98d-5f3e5413aee6 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:10.942797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1255/2490] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:08:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:08:14.844397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:20.937599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:36.325068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:36.327552 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:36.433224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e315f-8721-466c-a0fd-8752ab4a93bf service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:36.438935 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:36.439902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:36.440183 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:36.440473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:36.442365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:36.455164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:36.458399 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:36.471060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1236/2491] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:08:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:08:38.328897 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0e76-d9d2-4b57-b4e8-431784a27956 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:38.329511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0e76-d9d2-4b57-b4e8-431784a27956 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:38.329511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0e76-d9d2-4b57-b4e8-431784a27956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:38.348898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-477a0e76-d9d2-4b57-b4e8-431784a27956 None None] MFA Rules not processed for user `2e9707561b4a4811bd7a7d3fed4fcd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:38.351729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-477a0e76-d9d2-4b57-b4e8-431784a27956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:38.433894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1256/2492] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:46.466209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=41 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:47.590322 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:47.663057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd452b70-3992-4a5c-982b-670e37658b08 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:47.668794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:47.669700 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:47.670034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:47.670296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] PATH_INFO: `/v3/users/4e521f459d8c4a3dbad8c6cee7f1fa42` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:47.671875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e521f459d8c4a3dbad8c6cee7f1fa42)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:47.673855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:47.921828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e521f459d8c4a3dbad8c6cee7f1fa42'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:47.922598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e521f459d8c4a3dbad8c6cee7f1fa42'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:48.034089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e521f459d8c4a3dbad8c6cee7f1fa42'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:48.042879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e521f459d8c4a3dbad8c6cee7f1fa42'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:48.043758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e521f459d8c4a3dbad8c6cee7f1fa42'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:48.071443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1237/2493] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:08:47 2022] DELETE /identity/v3/users/4e521f459d8c4a3dbad8c6cee7f1fa42 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:48.074223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77b22d-9fbd-463f-8b71-c6c913bdff24 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:48.074690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77b22d-9fbd-463f-8b71-c6c913bdff24 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:48.074785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77b22d-9fbd-463f-8b71-c6c913bdff24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:48.096766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-aa77b22d-9fbd-463f-8b71-c6c913bdff24 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:48.096766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-aa77b22d-9fbd-463f-8b71-c6c913bdff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:48.182859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1257/2494] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:08:48.194996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:48.196598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:48.265222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c265e1b-c3af-4805-90c3-3193dbac2ce0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:48.269110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c8fc9-fd3d-4f26-a7e6-0d90d24a7c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:48.269764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c8fc9-fd3d-4f26-a7e6-0d90d24a7c71 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:48.270053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c8fc9-fd3d-4f26-a7e6-0d90d24a7c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:48.270335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c8fc9-fd3d-4f26-a7e6-0d90d24a7c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:48.290054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c8fc9-fd3d-4f26-a7e6-0d90d24a7c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:48.291972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c8fc9-fd3d-4f26-a7e6-0d90d24a7c71 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:48.310567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1238/2495] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:08:48.801483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77b22d-9fbd-463f-8b71-c6c913bdff24 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:48.802941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-aa77b22d-9fbd-463f-8b71-c6c913bdff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:48.816192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77b22d-9fbd-463f-8b71-c6c913bdff24 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:48.821172 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:48.822068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:48.822381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:48.822660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] PATH_INFO: `/v3/projects/a4b6d1b30915472c876836b1500b3521` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:48.823890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b6d1b30915472c876836b1500b3521)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:48.825882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:49.007218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b6d1b30915472c876836b1500b3521', 'request_id': 'req-d232aa10-eed0-49a6-b700-dec5e67248c4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.014231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b6d1b30915472c876836b1500b3521', 'request_id': 'req-d232aa10-eed0-49a6-b700-dec5e67248c4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.108670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b6d1b30915472c876836b1500b3521', 'request_id': 'req-d232aa10-eed0-49a6-b700-dec5e67248c4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.127176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] The token cache is being invalidate because project a4b6d1b30915472c876836b1500b3521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:08:49.127406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.128096 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.129709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1258/2496] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:08:48 2022] DELETE /identity/v3/projects/a4b6d1b30915472c876836b1500b3521 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:49.133986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c8fc9-fd3d-4f26-a7e6-0d90d24a7c71 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:49.144570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c8fc9-fd3d-4f26-a7e6-0d90d24a7c71 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:49.178194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:49.232137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:49.233038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:49.233309 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:49.233588 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] PATH_INFO: `/v3/users/cae23d8b355b47f5b9a1e755a5c48250` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:49.234816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae23d8b355b47f5b9a1e755a5c48250)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:49.236846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:49.426061 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae23d8b355b47f5b9a1e755a5c48250'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.426894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae23d8b355b47f5b9a1e755a5c48250'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.508278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae23d8b355b47f5b9a1e755a5c48250'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.517331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae23d8b355b47f5b9a1e755a5c48250'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.518086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0b3061d3-427f-46ca-9eca-e0f168d7f74e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae23d8b355b47f5b9a1e755a5c48250'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:49.537685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1239/2497] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:08:49 2022] DELETE /identity/v3/users/cae23d8b355b47f5b9a1e755a5c48250 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:50.096975 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:50.098477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:50.120741 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232aa10-eed0-49a6-b700-dec5e67248c4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:50.173525 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:50.225145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:50.226655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:50.227134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:50.227694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] PATH_INFO: `/v3/projects/fc75a91eab9940468f8b57dd4a871f46` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:50.229941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc75a91eab9940468f8b57dd4a871f46)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:50.232436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:50.413183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc75a91eab9940468f8b57dd4a871f46', 'request_id': 'req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:50.422665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc75a91eab9940468f8b57dd4a871f46', 'request_id': 'req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:50.508272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc75a91eab9940468f8b57dd4a871f46', 'request_id': 'req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:50.527902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] The token cache is being invalidate because project fc75a91eab9940468f8b57dd4a871f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:08:50.528317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:50.529614 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5eaa9c65-1e80-4657-92eb-c7f0525a23a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:08:50.532026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1259/2498] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:08:50 2022] DELETE /identity/v3/projects/fc75a91eab9940468f8b57dd4a871f46 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:50.537283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b8d6-def9-4268-8891-4f079900f9d5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:50.538101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b8d6-def9-4268-8891-4f079900f9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:50.538101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b8d6-def9-4268-8891-4f079900f9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:50.561280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0ef7b8d6-def9-4268-8891-4f079900f9d5 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:50.563680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0ef7b8d6-def9-4268-8891-4f079900f9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:50.681345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1240/2499] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:50.693452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3b27-bcc9-47de-9cd4-b23d4803f60d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:50.693747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3b27-bcc9-47de-9cd4-b23d4803f60d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:50.694026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3b27-bcc9-47de-9cd4-b23d4803f60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:50.714889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-80cc3b27-bcc9-47de-9cd4-b23d4803f60d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:50.716159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-80cc3b27-bcc9-47de-9cd4-b23d4803f60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:50.724549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1260/2500] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:50.729082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7b8d6-def9-4268-8891-4f079900f9d5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:50.739801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7b8d6-def9-4268-8891-4f079900f9d5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:50.745580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:50.746358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:50.746655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:50.746925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:50.747708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:08:50.748015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:50.749710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:50.765574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1241/2501] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:08:50.772840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc3b27-bcc9-47de-9cd4-b23d4803f60d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:50.789468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc3b27-bcc9-47de-9cd4-b23d4803f60d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:50.797526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f8c65-25f9-4dd7-bcdb-002bfcdbcc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:50.798673 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f8c65-25f9-4dd7-bcdb-002bfcdbcc39 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:50.799170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f8c65-25f9-4dd7-bcdb-002bfcdbcc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:50.799706 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f8c65-25f9-4dd7-bcdb-002bfcdbcc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:50.800970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f8c65-25f9-4dd7-bcdb-002bfcdbcc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:50.804757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f8c65-25f9-4dd7-bcdb-002bfcdbcc39 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:50.989820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1261/2502] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:08:50 2022] POST /identity/v3/projects => generated 371 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:08:50.994494 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:51.005872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1a91d-ed1d-43b4-9e11-58c0fd0315e6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:51.034597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:51.075573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4513e-006c-4d15-996c-7ba45ecb21d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:51.076278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4513e-006c-4d15-996c-7ba45ecb21d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:51.076560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4513e-006c-4d15-996c-7ba45ecb21d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:51.076841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4513e-006c-4d15-996c-7ba45ecb21d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:51.077730 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4513e-006c-4d15-996c-7ba45ecb21d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:51.079801 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4513e-006c-4d15-996c-7ba45ecb21d6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:51.239522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1242/2503] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:08:50 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:08:51.245938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f8c65-25f9-4dd7-bcdb-002bfcdbcc39 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:51.257354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f8c65-25f9-4dd7-bcdb-002bfcdbcc39 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:51.262600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3291a-958c-4e25-b698-dc1ceb93cb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:51.263325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3291a-958c-4e25-b698-dc1ceb93cb3a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:51.263925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3291a-958c-4e25-b698-dc1ceb93cb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:51.264415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3291a-958c-4e25-b698-dc1ceb93cb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:51.265851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3291a-958c-4e25-b698-dc1ceb93cb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:51.269303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3291a-958c-4e25-b698-dc1ceb93cb3a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:51.291865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1262/2504] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:08:51.297828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4513e-006c-4d15-996c-7ba45ecb21d6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:51.316960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4513e-006c-4d15-996c-7ba45ecb21d6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:51.325256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab8e3e-e518-4bb7-8c68-f5eed9b43ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:51.326501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab8e3e-e518-4bb7-8c68-f5eed9b43ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:51.327011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab8e3e-e518-4bb7-8c68-f5eed9b43ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:51.327496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab8e3e-e518-4bb7-8c68-f5eed9b43ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:51.328858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab8e3e-e518-4bb7-8c68-f5eed9b43ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:51.331983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab8e3e-e518-4bb7-8c68-f5eed9b43ef8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:51.353609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1243/2505] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:08:51.359150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3291a-958c-4e25-b698-dc1ceb93cb3a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:51.370275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3291a-958c-4e25-b698-dc1ceb93cb3a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:51.375418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:51.376329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:51.376610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:51.376890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] PATH_INFO: `/v3/projects/1f4a25da5f72426792020dcefce05a5b/users/0be2a07d278743f9b341eb4f413062ad/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:51.394346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4a25da5f72426792020dcefce05a5b, user_id=0be2a07d278743f9b341eb4f413062ad, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:51.397128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:51.483848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1263/2506] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:08:51 2022] PUT /identity/v3/projects/1f4a25da5f72426792020dcefce05a5b/users/0be2a07d278743f9b341eb4f413062ad/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:08:51.492429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e4f4-5082-485c-86cb-83107dba8ce0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:51.492429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e4f4-5082-485c-86cb-83107dba8ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:51.492429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e4f4-5082-485c-86cb-83107dba8ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:51.513760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5fa8e4f4-5082-485c-86cb-83107dba8ce0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:51.516012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5fa8e4f4-5082-485c-86cb-83107dba8ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:51.611143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1244/2507] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:51.622615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:51.624909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:51.697461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9e791-80f5-452e-8917-e4057655bbea admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:51.701966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3dd26-1bf4-4211-a3d1-006d7dfc605a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:51.702647 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3dd26-1bf4-4211-a3d1-006d7dfc605a service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:51.702921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3dd26-1bf4-4211-a3d1-006d7dfc605a service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:51.703195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3dd26-1bf4-4211-a3d1-006d7dfc605a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:51.713786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3dd26-1bf4-4211-a3d1-006d7dfc605a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:51.715799 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3dd26-1bf4-4211-a3d1-006d7dfc605a service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:51.720630 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:51.728154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1264/2508] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:08:53.431742 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac323ca-a7ae-4e93-b049-5318a4858bc4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:53.432312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac323ca-a7ae-4e93-b049-5318a4858bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:53.432738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac323ca-a7ae-4e93-b049-5318a4858bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:53.465121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0ac323ca-a7ae-4e93-b049-5318a4858bc4 None None] MFA Rules not processed for user `0be2a07d278743f9b341eb4f413062ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:53.466276 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0ac323ca-a7ae-4e93-b049-5318a4858bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:53.540050 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1245/2509] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:53.544921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3dd26-1bf4-4211-a3d1-006d7dfc605a service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:53.558463 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3dd26-1bf4-4211-a3d1-006d7dfc605a service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:53.563714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26afc14-dded-42ee-9f71-984a122902b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:53.564390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26afc14-dded-42ee-9f71-984a122902b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:53.564916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26afc14-dded-42ee-9f71-984a122902b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:53.565191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26afc14-dded-42ee-9f71-984a122902b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:53.565905 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26afc14-dded-42ee-9f71-984a122902b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:53.567991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26afc14-dded-42ee-9f71-984a122902b3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:53.686600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1265/2510] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:08:53 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:08:53.688200 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac323ca-a7ae-4e93-b049-5318a4858bc4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:53.706102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac323ca-a7ae-4e93-b049-5318a4858bc4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:53.774976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:53.807104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4e3fe-7e70-4e7c-98af-6e11875ebc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:53.808574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4e3fe-7e70-4e7c-98af-6e11875ebc71 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:53.808765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4e3fe-7e70-4e7c-98af-6e11875ebc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:53.809098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4e3fe-7e70-4e7c-98af-6e11875ebc71 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:53.809960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4e3fe-7e70-4e7c-98af-6e11875ebc71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:53.812254 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4e3fe-7e70-4e7c-98af-6e11875ebc71 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:53.986520 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1246/2511] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:08:53 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:08:53.988099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26afc14-dded-42ee-9f71-984a122902b3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:54.002376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26afc14-dded-42ee-9f71-984a122902b3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:54.009298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc7327-cb07-4dc5-b0eb-0949190d6dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:54.010158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7327-cb07-4dc5-b0eb-0949190d6dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:54.010407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7327-cb07-4dc5-b0eb-0949190d6dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:54.011146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7327-cb07-4dc5-b0eb-0949190d6dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:54.015460 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc7327-cb07-4dc5-b0eb-0949190d6dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:54.015460 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc7327-cb07-4dc5-b0eb-0949190d6dbc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:54.035408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1266/2512] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:08:54.040194 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4e3fe-7e70-4e7c-98af-6e11875ebc71 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:54.053014 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4e3fe-7e70-4e7c-98af-6e11875ebc71 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:54.057951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9364c3b-0faf-4ba3-bc2d-5b9d4b503977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:54.058896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9364c3b-0faf-4ba3-bc2d-5b9d4b503977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:54.059210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9364c3b-0faf-4ba3-bc2d-5b9d4b503977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:54.059512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9364c3b-0faf-4ba3-bc2d-5b9d4b503977 admin admin] PATH_INFO: `/v3/projects/44cab3655f454726bbbc4e42b2c50db7/users/90e79425886042d3b57a126cd5602747/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:54.075842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9364c3b-0faf-4ba3-bc2d-5b9d4b503977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cab3655f454726bbbc4e42b2c50db7, user_id=90e79425886042d3b57a126cd5602747, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:54.078600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9364c3b-0faf-4ba3-bc2d-5b9d4b503977 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:54.206928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1247/2513] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:08:54 2022] PUT /identity/v3/projects/44cab3655f454726bbbc4e42b2c50db7/users/90e79425886042d3b57a126cd5602747/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:54.211211 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc7327-cb07-4dc5-b0eb-0949190d6dbc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:54.224148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc7327-cb07-4dc5-b0eb-0949190d6dbc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:54.292081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dfdb2-630b-472b-a02c-6c6acee86a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:54.292762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dfdb2-630b-472b-a02c-6c6acee86a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:54.293043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dfdb2-630b-472b-a02c-6c6acee86a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:54.293322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dfdb2-630b-472b-a02c-6c6acee86a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:54.294141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dfdb2-630b-472b-a02c-6c6acee86a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:54.295957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dfdb2-630b-472b-a02c-6c6acee86a73 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:54.312866 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1267/2514] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:08:54.315976 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9364c3b-0faf-4ba3-bc2d-5b9d4b503977 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:54.327049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9364c3b-0faf-4ba3-bc2d-5b9d4b503977 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:54.332080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:54.333001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:54.337563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:54.337563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] PATH_INFO: `/v3/projects/44cab3655f454726bbbc4e42b2c50db7/users/90e79425886042d3b57a126cd5602747/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:54.337563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cab3655f454726bbbc4e42b2c50db7, user_id=90e79425886042d3b57a126cd5602747, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:54.344627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:54.354526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1248/2515] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:08:54 2022] PUT /identity/v3/projects/44cab3655f454726bbbc4e42b2c50db7/users/90e79425886042d3b57a126cd5602747/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:08:56.222744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1577c84-1243-4e40-b1d4-366ede7ef6b5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:56.223480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1577c84-1243-4e40-b1d4-366ede7ef6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:56.223480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1577c84-1243-4e40-b1d4-366ede7ef6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:56.263431 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e1577c84-1243-4e40-b1d4-366ede7ef6b5 None None] MFA Rules not processed for user `90e79425886042d3b57a126cd5602747`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:56.265132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e1577c84-1243-4e40-b1d4-366ede7ef6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:56.378331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1268/2516] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:56.392098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:56.393947 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:56.488560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a13de9-8bd6-4464-9a6f-51dc2f5a41d9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:56.494666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e431c-5dcf-40c5-bab0-8106a4d52357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:56.495533 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e431c-5dcf-40c5-bab0-8106a4d52357 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:56.496298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e431c-5dcf-40c5-bab0-8106a4d52357 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:56.496589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e431c-5dcf-40c5-bab0-8106a4d52357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:56.580099 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e431c-5dcf-40c5-bab0-8106a4d52357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:56.582051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e431c-5dcf-40c5-bab0-8106a4d52357 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:56.594027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1249/2517] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:08:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:08:56.883749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a634b-7b79-4aee-af82-3f441caee1c3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:56.884053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a634b-7b79-4aee-af82-3f441caee1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:56.884329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a634b-7b79-4aee-af82-3f441caee1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:56.907021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-336a634b-7b79-4aee-af82-3f441caee1c3 None None] MFA Rules not processed for user `8622ec5212c84d87b9b7c8f725012a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:56.909202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-336a634b-7b79-4aee-af82-3f441caee1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:56.968826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:08:56.989810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1269/2518] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:08:59.578809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e431c-5dcf-40c5-bab0-8106a4d52357 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:08:59.591959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e431c-5dcf-40c5-bab0-8106a4d52357 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:08:59.596126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:08:59.597496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:59.597496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:59.597496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:59.608504 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:08:59.609980 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:08:59.632967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1250/2519] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:08:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:08:59.634272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8c8cb-0683-4395-a9e1-b4c22cdb45d9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:08:59.636262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8c8cb-0683-4395-a9e1-b4c22cdb45d9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:08:59.636814 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8c8cb-0683-4395-a9e1-b4c22cdb45d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:08:59.670061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c5f8c8cb-0683-4395-a9e1-b4c22cdb45d9 None None] MFA Rules not processed for user `021817b0eef94fc2a19d90c511ec9298`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:08:59.673002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c5f8c8cb-0683-4395-a9e1-b4c22cdb45d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:08:59.750136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1270/2520] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:08:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:09:05.802490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:05.804468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:05.882800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:05.929737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619e83b-43e4-45e8-a654-fcb03b52c8be service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:05.937271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:05.938484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:05.939006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:05.939841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:05.943417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:06.019716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:06.021385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:06.044554 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1251/2521] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:09:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:09:09.741214 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=29 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:16.193802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=36 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:17.015282 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8c8cb-0683-4395-a9e1-b4c22cdb45d9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:17.093270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8c8cb-0683-4395-a9e1-b4c22cdb45d9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:17.099165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:17.100455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:17.100812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:17.101165 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] PATH_INFO: `/v3/users/0be2a07d278743f9b341eb4f413062ad` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:17.102530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be2a07d278743f9b341eb4f413062ad)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:17.105322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:17.346539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be2a07d278743f9b341eb4f413062ad'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:17.347153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be2a07d278743f9b341eb4f413062ad'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:17.347807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be2a07d278743f9b341eb4f413062ad'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:17.361114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be2a07d278743f9b341eb4f413062ad'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:17.370594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be2a07d278743f9b341eb4f413062ad'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:17.438829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1271/2522] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:09:17 2022] DELETE /identity/v3/users/0be2a07d278743f9b341eb4f413062ad => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:09:17.830030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:17.840240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8152-27e0-451a-a38c-a42655d051cf service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:17.915370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:17.916240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:17.917137 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:17.917423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] PATH_INFO: `/v3/projects/1f4a25da5f72426792020dcefce05a5b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:17.918671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4a25da5f72426792020dcefce05a5b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:17.920631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:18.069592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4a25da5f72426792020dcefce05a5b', 'request_id': 'req-bb541479-c058-4c61-a889-a0db01fb50c7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.075088 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4a25da5f72426792020dcefce05a5b', 'request_id': 'req-bb541479-c058-4c61-a889-a0db01fb50c7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.082780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4a25da5f72426792020dcefce05a5b', 'request_id': 'req-bb541479-c058-4c61-a889-a0db01fb50c7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.178712 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] The token cache is being invalidate because project 1f4a25da5f72426792020dcefce05a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:09:18.180853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.180853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb541479-c058-4c61-a889-a0db01fb50c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.181217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1252/2523] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:09:17 2022] DELETE /identity/v3/projects/1f4a25da5f72426792020dcefce05a5b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:09:18.185810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:18.187304 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:18.200870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab96d76-cb14-4902-a823-a4491312a7f2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:18.260856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:18.262359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:18.262644 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:18.262914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] PATH_INFO: `/v3/users/90e79425886042d3b57a126cd5602747` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:18.264152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e79425886042d3b57a126cd5602747)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:18.266098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:18.292276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:18.460253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e79425886042d3b57a126cd5602747'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.460907 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e79425886042d3b57a126cd5602747'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.461508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e79425886042d3b57a126cd5602747'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.474088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e79425886042d3b57a126cd5602747'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.481470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e79425886042d3b57a126cd5602747'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:18.532513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a607221-0afd-49ca-9eef-509b25be0cad None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:18.532795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a607221-0afd-49ca-9eef-509b25be0cad None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:18.533081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a607221-0afd-49ca-9eef-509b25be0cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:18.552028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3a607221-0afd-49ca-9eef-509b25be0cad None None] MFA Rules not processed for user `708734ba3ddc49a3a9dba1d891aeccc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:09:18.553262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3a607221-0afd-49ca-9eef-509b25be0cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:18.559960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1272/2524] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:09:18 2022] DELETE /identity/v3/users/90e79425886042d3b57a126cd5602747 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:09:18.614020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1253/2525] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:09:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:09:18.805659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:18.807328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:18.891160 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90782b-610e-4388-ba9b-3cc661dc24df admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:18.896166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:18.897038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:18.897322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:18.897607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] PATH_INFO: `/v3/projects/44cab3655f454726bbbc4e42b2c50db7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:18.899091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44cab3655f454726bbbc4e42b2c50db7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:18.901825 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:19.024331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44cab3655f454726bbbc4e42b2c50db7', 'request_id': 'req-c05edbfc-68cf-480c-bdf6-aeea515993c0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:19.031818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44cab3655f454726bbbc4e42b2c50db7', 'request_id': 'req-c05edbfc-68cf-480c-bdf6-aeea515993c0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:19.101221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44cab3655f454726bbbc4e42b2c50db7', 'request_id': 'req-c05edbfc-68cf-480c-bdf6-aeea515993c0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:19.114321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] The token cache is being invalidate because project 44cab3655f454726bbbc4e42b2c50db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:09:19.114321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:19.115091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c05edbfc-68cf-480c-bdf6-aeea515993c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:09:19.116763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1273/2526] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:09:18 2022] DELETE /identity/v3/projects/44cab3655f454726bbbc4e42b2c50db7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:09:19.123564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf1dd-b0b2-4e54-9eca-777d06e7638b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.127503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf1dd-b0b2-4e54-9eca-777d06e7638b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.127503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf1dd-b0b2-4e54-9eca-777d06e7638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.144929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-159cf1dd-b0b2-4e54-9eca-777d06e7638b None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:09:19.146155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-159cf1dd-b0b2-4e54-9eca-777d06e7638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:19.190403 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:19.217704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1254/2527] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:09:19.226124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3450f-7681-4967-ae9a-01408f6481d6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.230362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3450f-7681-4967-ae9a-01408f6481d6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.230362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3450f-7681-4967-ae9a-01408f6481d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.246759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c9c3450f-7681-4967-ae9a-01408f6481d6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:09:19.248136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c9c3450f-7681-4967-ae9a-01408f6481d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:19.256242 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1274/2528] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:09:19.262957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cf1dd-b0b2-4e54-9eca-777d06e7638b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:19.272943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cf1dd-b0b2-4e54-9eca-777d06e7638b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:19.278213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:19.278912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.279284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.279510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.280321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:09:19.280607 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:19.282235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:19.294472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1255/2529] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:09:19.298890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3450f-7681-4967-ae9a-01408f6481d6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:19.308964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3450f-7681-4967-ae9a-01408f6481d6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:19.313775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a2382-98ac-494e-ba81-f697abd76efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:19.314424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a2382-98ac-494e-ba81-f697abd76efe admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.314996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a2382-98ac-494e-ba81-f697abd76efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.315280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a2382-98ac-494e-ba81-f697abd76efe admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.315969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a2382-98ac-494e-ba81-f697abd76efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:19.317932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a2382-98ac-494e-ba81-f697abd76efe admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:19.429283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:19.441555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1275/2530] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:09:19 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:09:19.443080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:19.456876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9c7c9-b55d-4e6d-aa1f-03e80d69022e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:19.516618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f0ac-2ac0-4111-9359-ac61fa25dc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:19.517301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f0ac-2ac0-4111-9359-ac61fa25dc89 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.517581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f0ac-2ac0-4111-9359-ac61fa25dc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.517910 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f0ac-2ac0-4111-9359-ac61fa25dc89 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.518604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464f0ac-2ac0-4111-9359-ac61fa25dc89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:19.520545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464f0ac-2ac0-4111-9359-ac61fa25dc89 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:19.644692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1256/2531] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:09:19 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:09:19.649081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a2382-98ac-494e-ba81-f697abd76efe admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:19.659714 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a2382-98ac-494e-ba81-f697abd76efe admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:19.664477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f52ba-a494-4e73-a8d8-174752280796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:19.665141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f52ba-a494-4e73-a8d8-174752280796 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.665421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f52ba-a494-4e73-a8d8-174752280796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.665789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f52ba-a494-4e73-a8d8-174752280796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.666541 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f52ba-a494-4e73-a8d8-174752280796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:19.668677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f52ba-a494-4e73-a8d8-174752280796 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:19.679281 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1276/2532] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:09:19.684273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f0ac-2ac0-4111-9359-ac61fa25dc89 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:19.694715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f0ac-2ac0-4111-9359-ac61fa25dc89 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:19.700186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af9f96-e26e-4832-955f-b0b647baff7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:19.700861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af9f96-e26e-4832-955f-b0b647baff7c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.701148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af9f96-e26e-4832-955f-b0b647baff7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.701420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af9f96-e26e-4832-955f-b0b647baff7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.702162 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af9f96-e26e-4832-955f-b0b647baff7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:19.704259 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af9f96-e26e-4832-955f-b0b647baff7c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:19.718503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1257/2533] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:09:19.723073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f52ba-a494-4e73-a8d8-174752280796 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:19.733816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f52ba-a494-4e73-a8d8-174752280796 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:19.738593 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:19.739554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.739980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.740268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] PATH_INFO: `/v3/projects/6c1ef6ecf6524e5fbfd03701322fb85b/users/141fca3ac956419f8f1ad65163b47005/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.756051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1ef6ecf6524e5fbfd03701322fb85b, user_id=141fca3ac956419f8f1ad65163b47005, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:19.758755 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:19.909186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1277/2534] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:09:19 2022] PUT /identity/v3/projects/6c1ef6ecf6524e5fbfd03701322fb85b/users/141fca3ac956419f8f1ad65163b47005/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:09:19.920248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd716-3d6f-4f94-8ab5-20778996b25d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:19.920581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd716-3d6f-4f94-8ab5-20778996b25d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:19.920852 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd716-3d6f-4f94-8ab5-20778996b25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:19.949753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-44fdd716-3d6f-4f94-8ab5-20778996b25d None None] MFA Rules not processed for user `141fca3ac956419f8f1ad65163b47005`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:09:19.951148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-44fdd716-3d6f-4f94-8ab5-20778996b25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:20.020209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1258/2535] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:09:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:09:20.033802 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:20.035602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:20.096661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fa17c-2ca4-444a-b82b-9764f426cfd8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:20.100526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8327d9916424843b2a67cbdd856ad37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:20.101181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:20.101500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:20.101779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:20.113853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:20.115403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:20.125951 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1278/2536] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:09:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:09:28.702015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdd716-3d6f-4f94-8ab5-20778996b25d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:28.703642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-44fdd716-3d6f-4f94-8ab5-20778996b25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:28.739738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:28.764063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdd716-3d6f-4f94-8ab5-20778996b25d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:28.767837 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:28.768501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:28.768788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:28.769070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:28.770880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:28.845897 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:28.847404 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:28.858593 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1259/2537] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:09:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:09:29.983010 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:29.984546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:29.997826 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743429fc-9b63-45db-a3a9-37d0c0e727e4 service glance] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:30.001452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:30.002095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:30.002375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:30.002646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:30.004458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:30.018366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:30.019823 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:30.031328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1279/2538] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:09:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:09:38.853806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=40 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:40.025956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=47,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:09:54.073996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:09:54.075481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:54.133874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ec13-c9f7-492d-9b33-a8b7f5e276c9 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:09:54.137561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed80fe-651d-4ea0-b3be-e6cc62eedf00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:09:54.138419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed80fe-651d-4ea0-b3be-e6cc62eedf00 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:09:54.138693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed80fe-651d-4ea0-b3be-e6cc62eedf00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:09:54.138966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed80fe-651d-4ea0-b3be-e6cc62eedf00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:09:54.141369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-49ed80fe-651d-4ea0-b3be-e6cc62eedf00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:09:54.200671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed80fe-651d-4ea0-b3be-e6cc62eedf00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:09:54.202754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed80fe-651d-4ea0-b3be-e6cc62eedf00 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:09:54.211987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1260/2539] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:09:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:04.207703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=48 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:07.864357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:07.935699 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a360c-9437-43a2-bac5-cef57ef82c73 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:07.941173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:07.942080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:07.942364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:07.942644 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] PATH_INFO: `/v3/users/141fca3ac956419f8f1ad65163b47005` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:07.943960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=141fca3ac956419f8f1ad65163b47005)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:07.945923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:08.368536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '141fca3ac956419f8f1ad65163b47005'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:08.369152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '141fca3ac956419f8f1ad65163b47005'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:08.369764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '141fca3ac956419f8f1ad65163b47005'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:08.384508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '141fca3ac956419f8f1ad65163b47005'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:08.392694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '141fca3ac956419f8f1ad65163b47005'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:08.489458 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1280/2540] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:10:07 2022] DELETE /identity/v3/users/141fca3ac956419f8f1ad65163b47005 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:10:08.494942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d88da-afd0-4972-8d18-5ff4e6719459 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:08.495877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d88da-afd0-4972-8d18-5ff4e6719459 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:08.496056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d88da-afd0-4972-8d18-5ff4e6719459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:08.519419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d86d88da-afd0-4972-8d18-5ff4e6719459 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:10:08.520952 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d86d88da-afd0-4972-8d18-5ff4e6719459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:08.588871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1261/2541] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:10:08.600923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:08.603403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:08.671319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d10cd-2713-4420-8107-24c0f6892491 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:08.676123 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:08.676901 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:08.677388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:08.677685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:08.691829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:08.693717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:08.708026 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1281/2542] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:09.480331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d88da-afd0-4972-8d18-5ff4e6719459 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:09.483534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d86d88da-afd0-4972-8d18-5ff4e6719459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:09.499699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d88da-afd0-4972-8d18-5ff4e6719459 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:09.505776 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:09.507047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:09.507706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:09.508467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] PATH_INFO: `/v3/projects/6c1ef6ecf6524e5fbfd03701322fb85b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:09.511334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1ef6ecf6524e5fbfd03701322fb85b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:09.515279 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:09.704269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:09.706297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:09.713025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1ef6ecf6524e5fbfd03701322fb85b', 'request_id': 'req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:09.725498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1ef6ecf6524e5fbfd03701322fb85b', 'request_id': 'req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:09.766822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:09.785497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1ef6ecf6524e5fbfd03701322fb85b', 'request_id': 'req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:09.906604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] The token cache is being invalidate because project 6c1ef6ecf6524e5fbfd03701322fb85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:10:09.907472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:09.910078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:09.913677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1262/2543] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:10:09 2022] DELETE /identity/v3/projects/6c1ef6ecf6524e5fbfd03701322fb85b => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:09.960181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8a622-8152-4a78-bbc5-3309c9ab5618 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:10.223540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:10.230226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:10.239884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:10.241029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:10.261092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:10.305356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:10.307603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:10.329358 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1282/2544] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:10:10.440143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:10.442664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:10.523613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71eb6-24f3-426f-8d3e-cc3efb63d8cf admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:10.527955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:10.528654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:10.528936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:10.529237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:10.532859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:10.549009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:10.552378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:10.562821 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1263/2545] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:12.196519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:12.199834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:12.226712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1f443-0fdb-4545-b384-386dcc65ed74 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:12.239085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:12.240724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:12.241697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:12.242429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:12.246698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:12.272492 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:12.276351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:12.301845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1283/2546] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:18.604025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:18.605523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:18.640023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:18.682990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7010fb3-5cfb-48a0-b215-32003028e92a service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:18.687800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:18.688737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:18.689044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:18.689300 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] PATH_INFO: `/v3/users/021817b0eef94fc2a19d90c511ec9298` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:18.690783 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] RBAC: Authorizing `identity:delete_user(user_id=021817b0eef94fc2a19d90c511ec9298)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:18.699426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:18.982508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021817b0eef94fc2a19d90c511ec9298'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:18.983196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021817b0eef94fc2a19d90c511ec9298'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.100767 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021817b0eef94fc2a19d90c511ec9298'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.114186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021817b0eef94fc2a19d90c511ec9298'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.115386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-82784a36-4ecf-476f-a414-2f88ec67b02d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021817b0eef94fc2a19d90c511ec9298'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.139943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1264/2547] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:10:18 2022] DELETE /identity/v3/users/021817b0eef94fc2a19d90c511ec9298 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:19.400088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:19.418884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e3deb-24f9-42e6-ab5b-b15ddd12e4af service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:19.482748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:19.483632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:19.483919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:19.484188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] PATH_INFO: `/v3/projects/e87ca5d9f50847d6ab501ca3a51ab37c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:19.485380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e87ca5d9f50847d6ab501ca3a51ab37c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:19.487390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:19.601413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:19.611405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e87ca5d9f50847d6ab501ca3a51ab37c', 'request_id': 'req-c721c623-49c8-4f92-aa26-45f53075e555'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.617032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e87ca5d9f50847d6ab501ca3a51ab37c', 'request_id': 'req-c721c623-49c8-4f92-aa26-45f53075e555'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.694621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e87ca5d9f50847d6ab501ca3a51ab37c', 'request_id': 'req-c721c623-49c8-4f92-aa26-45f53075e555'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.709317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] The token cache is being invalidate because project e87ca5d9f50847d6ab501ca3a51ab37c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:10:19.709609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.710878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:19.712107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1284/2548] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:10:19 2022] DELETE /identity/v3/projects/e87ca5d9f50847d6ab501ca3a51ab37c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:23.916439 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494271-57ac-46d7-8d06-3594c63fd03e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:23.917035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494271-57ac-46d7-8d06-3594c63fd03e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:23.917035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494271-57ac-46d7-8d06-3594c63fd03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:23.938349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b2494271-57ac-46d7-8d06-3594c63fd03e None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:10:23.940367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b2494271-57ac-46d7-8d06-3594c63fd03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:24.000833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1265/2549] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 1 04:10:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:10:24.202097 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:24.203618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:24.281429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721c623-49c8-4f92-aa26-45f53075e555 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:24.286927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:24.287624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:24.288013 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:24.288203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:24.298632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:24.300718 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:24.309781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1285/2550] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:10:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:10:24.860654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2494271-57ac-46d7-8d06-3594c63fd03e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:24.864420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b2494271-57ac-46d7-8d06-3594c63fd03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:24.924894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2494271-57ac-46d7-8d06-3594c63fd03e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:24.929204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:24.929766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:24.930051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:24.930328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:24.932247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:24.938597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:25.004356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:25.005961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:25.017850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1266/2551] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:10:25.862520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:25.865210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:25.961436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c06091-411d-4dc5-950c-c35ad86f5537 service swift] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:25.967956 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:25.968641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:25.968921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:25.969197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:25.971102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:26.040183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:26.055027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:26.056882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:26.067451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1286/2552] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:30.571060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:30.573127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:30.663785 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1451b7-8d62-45c8-a4d3-87fcca00a015 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:30.669130 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:30.669829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:30.670196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:30.670478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:30.672333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:30.683948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:30.685573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:30.696926 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1267/2553] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:30.748532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:30.750135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:30.770967 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599b0b7-97c2-4816-985a-f4e4be35bcd0 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:30.775736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:30.776797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:30.777081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:30.777364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] PATH_INFO: `/v3/users/8622ec5212c84d87b9b7c8f725012a07` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:30.778599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8622ec5212c84d87b9b7c8f725012a07)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:30.781303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:31.084781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8622ec5212c84d87b9b7c8f725012a07'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.085461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8622ec5212c84d87b9b7c8f725012a07'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.086059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8622ec5212c84d87b9b7c8f725012a07'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.100868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8622ec5212c84d87b9b7c8f725012a07'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.108763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6a9298e-805d-4552-8847-c24478cd20b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8622ec5212c84d87b9b7c8f725012a07'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.203559 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1287/2554] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:10:30 2022] DELETE /identity/v3/users/8622ec5212c84d87b9b7c8f725012a07 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:31.490996 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:31.503965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dd913-28db-482f-ac6d-093330a307e5 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:31.567495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:31.568503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:31.568789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:31.569066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] PATH_INFO: `/v3/projects/f0143140d4f649478e36abcee284640c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:31.570256 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0143140d4f649478e36abcee284640c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:31.572188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:31.667873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:31.691741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0143140d4f649478e36abcee284640c', 'request_id': 'req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.698592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0143140d4f649478e36abcee284640c', 'request_id': 'req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.705858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0143140d4f649478e36abcee284640c', 'request_id': 'req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.808248 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] The token cache is being invalidate because project f0143140d4f649478e36abcee284640c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:10:31.808858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.809924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3f09dd1-914e-4082-82b5-9a4d42aa0dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:31.812800 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1268/2555] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:10:31 2022] DELETE /identity/v3/projects/f0143140d4f649478e36abcee284640c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:31.818677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06b6eb-34c1-4dc2-a926-2006ffdb2cfa None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:31.819578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06b6eb-34c1-4dc2-a926-2006ffdb2cfa None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:31.820033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06b6eb-34c1-4dc2-a926-2006ffdb2cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:31.839592 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bd06b6eb-34c1-4dc2-a926-2006ffdb2cfa None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:10:31.840763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bd06b6eb-34c1-4dc2-a926-2006ffdb2cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:31.904854 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1288/2556] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:10:31.912553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfac5f9-9403-41e2-b689-2228c0d0d606 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:31.912838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfac5f9-9403-41e2-b689-2228c0d0d606 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:31.913498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfac5f9-9403-41e2-b689-2228c0d0d606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:31.941133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1dfac5f9-9403-41e2-b689-2228c0d0d606 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:10:31.942478 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1dfac5f9-9403-41e2-b689-2228c0d0d606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:31.952132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1269/2557] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:10:31.961965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06b6eb-34c1-4dc2-a926-2006ffdb2cfa None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:31.972461 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06b6eb-34c1-4dc2-a926-2006ffdb2cfa None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:31.977274 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:31.978027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:31.978381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:31.978658 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:31.979423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:10:31.979733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:31.981347 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:31.997310 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1289/2558] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:10:31.998571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfac5f9-9403-41e2-b689-2228c0d0d606 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:32.008245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfac5f9-9403-41e2-b689-2228c0d0d606 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:32.013048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65032ba-c8fe-406d-96ce-ffe8f0bd5e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:32.013809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65032ba-c8fe-406d-96ce-ffe8f0bd5e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:32.014089 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65032ba-c8fe-406d-96ce-ffe8f0bd5e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:32.014365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65032ba-c8fe-406d-96ce-ffe8f0bd5e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:32.015090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65032ba-c8fe-406d-96ce-ffe8f0bd5e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:32.017107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65032ba-c8fe-406d-96ce-ffe8f0bd5e00 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:32.102111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1270/2559] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:10:31 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:10:32.106213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:32.116722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aa9de-2e84-4fb3-9a0e-02f46598d4ba admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:32.167129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:32.179645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49378-f28d-4e23-933f-c30e3cbe17bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:32.180325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49378-f28d-4e23-933f-c30e3cbe17bd admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:32.180610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49378-f28d-4e23-933f-c30e3cbe17bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:32.180888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49378-f28d-4e23-933f-c30e3cbe17bd admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:32.181594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a49378-f28d-4e23-933f-c30e3cbe17bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:32.183545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a49378-f28d-4e23-933f-c30e3cbe17bd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:32.274417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1290/2560] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:10:32 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:10:32.281526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65032ba-c8fe-406d-96ce-ffe8f0bd5e00 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:32.300269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65032ba-c8fe-406d-96ce-ffe8f0bd5e00 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:32.312097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3388f-15c5-43f0-ad2a-e078617fc90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:32.312998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3388f-15c5-43f0-ad2a-e078617fc90c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:32.318909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3388f-15c5-43f0-ad2a-e078617fc90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:32.318909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3388f-15c5-43f0-ad2a-e078617fc90c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:32.318909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3388f-15c5-43f0-ad2a-e078617fc90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:32.318909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3388f-15c5-43f0-ad2a-e078617fc90c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:32.331303 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1271/2561] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:10:32.339798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49378-f28d-4e23-933f-c30e3cbe17bd admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:32.352290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49378-f28d-4e23-933f-c30e3cbe17bd admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:32.358429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0860c0-bc6a-45b5-b101-fa6bf8b3463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:32.359391 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0860c0-bc6a-45b5-b101-fa6bf8b3463d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:32.359695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0860c0-bc6a-45b5-b101-fa6bf8b3463d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:32.359978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0860c0-bc6a-45b5-b101-fa6bf8b3463d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:32.360801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0860c0-bc6a-45b5-b101-fa6bf8b3463d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:32.362694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0860c0-bc6a-45b5-b101-fa6bf8b3463d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:32.381021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1291/2562] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:10:32.385893 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3388f-15c5-43f0-ad2a-e078617fc90c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:32.398056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3388f-15c5-43f0-ad2a-e078617fc90c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:32.405457 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:32.407342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:32.407342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:32.407342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] PATH_INFO: `/v3/projects/f76a684261dd49f68a42a4e157355f37/users/4ac71714265342098d2487a701a6c364/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:32.438490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76a684261dd49f68a42a4e157355f37, user_id=4ac71714265342098d2487a701a6c364, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:32.441896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:32.520790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1272/2563] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:10:32 2022] PUT /identity/v3/projects/f76a684261dd49f68a42a4e157355f37/users/4ac71714265342098d2487a701a6c364/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:32.526447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff17a-4ccd-4b15-909b-b96a6a3341bc None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:32.526751 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff17a-4ccd-4b15-909b-b96a6a3341bc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:32.526999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff17a-4ccd-4b15-909b-b96a6a3341bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:32.546054 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2fcff17a-4ccd-4b15-909b-b96a6a3341bc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:10:32.547452 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2fcff17a-4ccd-4b15-909b-b96a6a3341bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:32.614315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1292/2564] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:10:32.626397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:32.628395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:32.698752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dd0de-35b8-4388-9457-7008f6406243 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:32.703085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0af5-ba17-45f9-a17d-eda6e5380d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:32.703779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0af5-ba17-45f9-a17d-eda6e5380d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:32.704109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0af5-ba17-45f9-a17d-eda6e5380d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:32.704546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0af5-ba17-45f9-a17d-eda6e5380d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:32.715098 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff0af5-ba17-45f9-a17d-eda6e5380d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:32.716642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff0af5-ba17-45f9-a17d-eda6e5380d17 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:32.727328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1273/2565] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:35.991027 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff17a-4ccd-4b15-909b-b96a6a3341bc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:35.993722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2fcff17a-4ccd-4b15-909b-b96a6a3341bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:36.015294 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff17a-4ccd-4b15-909b-b96a6a3341bc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:36.021686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:36.023071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:36.023696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:36.024483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] PATH_INFO: `/v3/users/abc2e04754884398afc999d423e2a711` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:36.026266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc2e04754884398afc999d423e2a711)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:36.029286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:36.239734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc2e04754884398afc999d423e2a711'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:36.240532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc2e04754884398afc999d423e2a711'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:36.241517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc2e04754884398afc999d423e2a711'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:36.261710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc2e04754884398afc999d423e2a711'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:36.272082 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc2e04754884398afc999d423e2a711'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:36.362089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1293/2566] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:10:35 2022] DELETE /identity/v3/users/abc2e04754884398afc999d423e2a711 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:36.795953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0af5-ba17-45f9-a17d-eda6e5380d17 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:36.809087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0af5-ba17-45f9-a17d-eda6e5380d17 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:36.891025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:36.893233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:36.893233 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:36.893737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] PATH_INFO: `/v3/projects/2940ca71c4b24b5facc194b61d748a11` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:36.896263 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2940ca71c4b24b5facc194b61d748a11)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:36.899402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:36.903036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:37.062392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2940ca71c4b24b5facc194b61d748a11', 'request_id': 'req-6c2158b2-235c-4de6-a113-6d0259801bfb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.071044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2940ca71c4b24b5facc194b61d748a11', 'request_id': 'req-6c2158b2-235c-4de6-a113-6d0259801bfb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.079942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2940ca71c4b24b5facc194b61d748a11', 'request_id': 'req-6c2158b2-235c-4de6-a113-6d0259801bfb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.197563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] The token cache is being invalidate because project 2940ca71c4b24b5facc194b61d748a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:10:37.197885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.198687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.201296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1274/2567] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:10:36 2022] DELETE /identity/v3/projects/2940ca71c4b24b5facc194b61d748a11 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:37.206909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:37.208620 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:37.223474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406672cc-ab77-41ac-afef-e8d527b64dec admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:37.284255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:37.321940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:37.323916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:37.324424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:37.324948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] PATH_INFO: `/v3/users/2f12518be45d4e3382554f5d5c7d291c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:37.327758 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f12518be45d4e3382554f5d5c7d291c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:37.333388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:37.581667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f12518be45d4e3382554f5d5c7d291c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.582709 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f12518be45d4e3382554f5d5c7d291c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.583767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f12518be45d4e3382554f5d5c7d291c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.613607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f12518be45d4e3382554f5d5c7d291c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.642577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-321a60cd-4005-49db-98e0-c6e2e5915ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f12518be45d4e3382554f5d5c7d291c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:37.779788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1294/2568] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:10:37 2022] DELETE /identity/v3/users/2f12518be45d4e3382554f5d5c7d291c => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:10:38.213349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:38.215708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:38.246920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2158b2-235c-4de6-a113-6d0259801bfb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:38.328467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:38.329649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:38.330149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:38.330350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] PATH_INFO: `/v3/projects/8660d76489724a8f89de4df44cfd5bec` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:38.331721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8660d76489724a8f89de4df44cfd5bec)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:38.334813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:38.449792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8660d76489724a8f89de4df44cfd5bec', 'request_id': 'req-ebe32176-fea5-4e87-88da-0fd24e6c66e9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:38.457472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8660d76489724a8f89de4df44cfd5bec', 'request_id': 'req-ebe32176-fea5-4e87-88da-0fd24e6c66e9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:38.464818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8660d76489724a8f89de4df44cfd5bec', 'request_id': 'req-ebe32176-fea5-4e87-88da-0fd24e6c66e9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:38.633188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] The token cache is being invalidate because project 8660d76489724a8f89de4df44cfd5bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:10:38.633479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:38.634327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:10:38.635957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1275/2569] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:10:38 2022] DELETE /identity/v3/projects/8660d76489724a8f89de4df44cfd5bec => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:10:39.801255 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147295ff-c588-4a05-87cf-61cac4aca6e5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:39.801827 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147295ff-c588-4a05-87cf-61cac4aca6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:39.801955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147295ff-c588-4a05-87cf-61cac4aca6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:39.830051 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-147295ff-c588-4a05-87cf-61cac4aca6e5 None None] MFA Rules not processed for user `4ac71714265342098d2487a701a6c364`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:10:39.831304 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-147295ff-c588-4a05-87cf-61cac4aca6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:39.906754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1295/2570] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:10:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:10:39.922492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:39.924747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:39.957751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:40.013428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe32176-fea5-4e87-88da-0fd24e6c66e9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:40.020114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:40.020964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:40.021253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:40.021531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:40.032134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:40.033866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:40.060784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1276/2571] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:41.856139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eceb1-5e6b-42db-ad77-5673b211feae None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:41.858225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eceb1-5e6b-42db-ad77-5673b211feae None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:41.860286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eceb1-5e6b-42db-ad77-5673b211feae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:41.893860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ee4eceb1-5e6b-42db-ad77-5673b211feae None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:10:41.895098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ee4eceb1-5e6b-42db-ad77-5673b211feae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:41.950453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:41.969248 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1296/2572] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:10:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:10:41.985198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:41.986601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:42.131492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87d5cf-1c2b-4341-a31b-1a823f21dd08 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:42.140051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:42.141122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:42.141577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:42.141859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:42.165134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:42.168223 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:42.179153 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1277/2573] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:10:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 04:10:43.030067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90245-bfd7-485a-9f53-9da7efac1c8f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:43.037343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90245-bfd7-485a-9f53-9da7efac1c8f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:43.037343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90245-bfd7-485a-9f53-9da7efac1c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:43.072189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-57f90245-bfd7-485a-9f53-9da7efac1c8f None None] MFA Rules not processed for user `4ac71714265342098d2487a701a6c364`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:10:43.073694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-57f90245-bfd7-485a-9f53-9da7efac1c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:43.087575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1297/2574] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:10:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:10:43.537272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:43.543173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:43.695775 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380673c3-9a20-45d8-9293-790c228fa531 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:43.710016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f90245-bfd7-485a-9f53-9da7efac1c8f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:43.720449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:43.722232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:43.722681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:43.723151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:43.725334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:43.733137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f90245-bfd7-485a-9f53-9da7efac1c8f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:43.737310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:43.738986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:43.751018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1278/2575] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:43.752016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:43.752861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:43.753246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:43.753617 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:43.758217 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:43.777803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:43.781075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:43.818728 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1298/2576] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:44.317843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:10:44.321207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:44.430866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:44.484342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34723f-5c6c-42a8-b1c7-9b8bc2cbc6c7 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:10:44.491737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dc42a-c294-4ccf-ba05-f720c26d6b37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:10:44.492416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dc42a-c294-4ccf-ba05-f720c26d6b37 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:10:44.492706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dc42a-c294-4ccf-ba05-f720c26d6b37 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:10:44.492987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dc42a-c294-4ccf-ba05-f720c26d6b37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:10:44.494916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-396dc42a-c294-4ccf-ba05-f720c26d6b37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:10:44.508668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396dc42a-c294-4ccf-ba05-f720c26d6b37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:10:44.510496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396dc42a-c294-4ccf-ba05-f720c26d6b37 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:10:44.543720 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1279/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:10:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:10:53.810311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=12 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:10:54.522245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=15 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:11:00.407675 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:11:00.409298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:00.421742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2870b2c-636b-4080-934e-7d7dded277cd service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:11:00.426012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3268d-b66c-4665-a4d2-9c42bdd82587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:11:00.429221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3268d-b66c-4665-a4d2-9c42bdd82587 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:00.429221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3268d-b66c-4665-a4d2-9c42bdd82587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:00.429221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3268d-b66c-4665-a4d2-9c42bdd82587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:00.429221 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b1a3268d-b66c-4665-a4d2-9c42bdd82587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:00.440055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3268d-b66c-4665-a4d2-9c42bdd82587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:11:00.441864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3268d-b66c-4665-a4d2-9c42bdd82587 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:11:00.452093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1299/2578] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:11:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:11:00.791624 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e59a7-d135-4eef-beaa-7fd768b13c74 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:00.792213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e59a7-d135-4eef-beaa-7fd768b13c74 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:00.792313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e59a7-d135-4eef-beaa-7fd768b13c74 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:00.794518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1280/2579] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:11:00.801483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1ed73-3556-469c-8361-e7f0daa0091c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:00.802009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1ed73-3556-469c-8361-e7f0daa0091c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:00.802299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1ed73-3556-469c-8361-e7f0daa0091c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:00.825131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fce1ed73-3556-469c-8361-e7f0daa0091c None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:11:00.826428 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fce1ed73-3556-469c-8361-e7f0daa0091c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:00.836583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1300/2580] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:11:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:11:00.849443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e59a7-d135-4eef-beaa-7fd768b13c74 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:11:00.851325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8a5e59a7-d135-4eef-beaa-7fd768b13c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:00.928953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e59a7-d135-4eef-beaa-7fd768b13c74 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:11:00.932877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366de588-9af1-42af-8097-a08eb938f1d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8327d9916424843b2a67cbdd856ad37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:11:00.933530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de588-9af1-42af-8097-a08eb938f1d9 service glance] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:00.933809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de588-9af1-42af-8097-a08eb938f1d9 service glance] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:00.934102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de588-9af1-42af-8097-a08eb938f1d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:00.945727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366de588-9af1-42af-8097-a08eb938f1d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:11:00.947168 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366de588-9af1-42af-8097-a08eb938f1d9 service glance] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:11:00.958933 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1281/2581] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:11:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:11:01.154115 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03854edc-fca5-479f-abbc-7bd89458e0b2 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:01.154474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03854edc-fca5-479f-abbc-7bd89458e0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:01.154815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03854edc-fca5-479f-abbc-7bd89458e0b2 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:01.157210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1301/2582] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:11:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:11:01.164209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e2006-aca8-47c3-966d-70b7ff17b49d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:01.164209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e2006-aca8-47c3-966d-70b7ff17b49d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:01.164209 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e2006-aca8-47c3-966d-70b7ff17b49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:01.183946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4f6e2006-aca8-47c3-966d-70b7ff17b49d None None] MFA Rules not processed for user `0beff646206e446386d91537df9eaa67`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:11:01.191218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4f6e2006-aca8-47c3-966d-70b7ff17b49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:01.200101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1282/2583] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:11:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:11:01.206056 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03854edc-fca5-479f-abbc-7bd89458e0b2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:11:01.219915 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03854edc-fca5-479f-abbc-7bd89458e0b2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:11:01.224566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68c1f8-9f23-4b20-802c-6850663c4725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:11:01.225495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c1f8-9f23-4b20-802c-6850663c4725 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:01.226066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c1f8-9f23-4b20-802c-6850663c4725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:01.226575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c1f8-9f23-4b20-802c-6850663c4725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:01.237311 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c1f8-9f23-4b20-802c-6850663c4725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:11:01.239199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c1f8-9f23-4b20-802c-6850663c4725 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:11:01.248505 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1302/2584] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:11:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:11:01.345488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7976-afeb-4c26-a40c-27cb83615048 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:01.345934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7976-afeb-4c26-a40c-27cb83615048 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:01.346441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7976-afeb-4c26-a40c-27cb83615048 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:01.348178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1283/2585] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 1 04:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 04:11:01.355098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5772-a8b6-4942-9170-b4d88847e05b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:01.358919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5772-a8b6-4942-9170-b4d88847e05b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:01.358919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5772-a8b6-4942-9170-b4d88847e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:01.377098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-12ec5772-a8b6-4942-9170-b4d88847e05b None None] MFA Rules not processed for user `dbd31b58571b476bad12735ff7445d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:11:01.378362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-12ec5772-a8b6-4942-9170-b4d88847e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:01.391898 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1303/2586] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 1 04:11:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:11:01.404936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7976-afeb-4c26-a40c-27cb83615048 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:11:01.419492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7976-afeb-4c26-a40c-27cb83615048 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:11:01.423618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108c808-3277-4f61-be45-4dbdd3470648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:11:01.424534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108c808-3277-4f61-be45-4dbdd3470648 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:01.425008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108c808-3277-4f61-be45-4dbdd3470648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:01.425458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108c808-3277-4f61-be45-4dbdd3470648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:01.438013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108c808-3277-4f61-be45-4dbdd3470648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:11:01.439456 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108c808-3277-4f61-be45-4dbdd3470648 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:11:01.452475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1284/2587] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:11:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:11:03.663694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f43300-9a32-4231-9b96-9259bba8120a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:03.664176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f43300-9a32-4231-9b96-9259bba8120a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:03.664268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f43300-9a32-4231-9b96-9259bba8120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:03.683353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f8f43300-9a32-4231-9b96-9259bba8120a None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:11:03.685345 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f8f43300-9a32-4231-9b96-9259bba8120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:03.695603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1304/2588] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:11:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:11:03.706886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108c808-3277-4f61-be45-4dbdd3470648 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:11:03.719214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108c808-3277-4f61-be45-4dbdd3470648 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:11:03.724542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ac435-021a-45de-93f6-734b8d6513d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:11:03.725187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ac435-021a-45de-93f6-734b8d6513d9 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:03.725474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ac435-021a-45de-93f6-734b8d6513d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:03.725754 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ac435-021a-45de-93f6-734b8d6513d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:03.735440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ac435-021a-45de-93f6-734b8d6513d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:11:03.737027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ac435-021a-45de-93f6-734b8d6513d9 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:11:03.747534 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1285/2589] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:11:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:11:13.677771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=21 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:11:13.743546 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=41 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:11:19.504742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11509a-87b9-4597-83eb-5cf33b62feec None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:19.505247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11509a-87b9-4597-83eb-5cf33b62feec None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:19.505344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11509a-87b9-4597-83eb-5cf33b62feec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:19.525610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7b11509a-87b9-4597-83eb-5cf33b62feec None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:11:19.526848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7b11509a-87b9-4597-83eb-5cf33b62feec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:19.536168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1305/2590] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 1 04:11:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:11:19.552406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ac435-021a-45de-93f6-734b8d6513d9 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:11:19.563905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ac435-021a-45de-93f6-734b8d6513d9 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:11:19.570618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1ee7d-db6b-4b01-9b23-cbab1a2be9f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:11:19.571311 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1ee7d-db6b-4b01-9b23-cbab1a2be9f0 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:19.571600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1ee7d-db6b-4b01-9b23-cbab1a2be9f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:19.571880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1ee7d-db6b-4b01-9b23-cbab1a2be9f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:19.581804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1ee7d-db6b-4b01-9b23-cbab1a2be9f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:11:19.583427 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1ee7d-db6b-4b01-9b23-cbab1a2be9f0 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:11:19.592204 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1286/2591] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:11:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:11:29.518680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:11:29.588758 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:11:32.049578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5342285-9933-4e90-9889-485a65e1745a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:32.050064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5342285-9933-4e90-9889-485a65e1745a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:32.050178 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5342285-9933-4e90-9889-485a65e1745a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:32.068836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e5342285-9933-4e90-9889-485a65e1745a None None] MFA Rules not processed for user `2e9707561b4a4811bd7a7d3fed4fcd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:11:32.070105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e5342285-9933-4e90-9889-485a65e1745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:32.129414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1306/2592] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:11:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:11:42.121302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=25 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:11:50.570879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397035b-6e6e-4379-b3a5-0cf7fa974d85 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:11:50.571608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397035b-6e6e-4379-b3a5-0cf7fa974d85 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:11:50.571608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397035b-6e6e-4379-b3a5-0cf7fa974d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:11:50.589969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c397035b-6e6e-4379-b3a5-0cf7fa974d85 None None] MFA Rules not processed for user `4ac71714265342098d2487a701a6c364`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:11:50.591177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c397035b-6e6e-4379-b3a5-0cf7fa974d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:11:50.598724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1287/2593] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:11:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:12:00.585085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:12:03.604446 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5342285-9933-4e90-9889-485a65e1745a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:03.617009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5342285-9933-4e90-9889-485a65e1745a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:03.623398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:03.624075 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:03.624355 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:03.624633 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:03.627019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:03.724417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:03.726177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:03.739752 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1307/2594] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:12:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:12:08.192930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397035b-6e6e-4379-b3a5-0cf7fa974d85 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:08.194506 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c397035b-6e6e-4379-b3a5-0cf7fa974d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:08.213732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397035b-6e6e-4379-b3a5-0cf7fa974d85 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:08.218842 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:08.220049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:08.220330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:08.220599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] PATH_INFO: `/v3/users/708734ba3ddc49a3a9dba1d891aeccc7` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:08.222071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=708734ba3ddc49a3a9dba1d891aeccc7)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:08.224045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:08.440895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '708734ba3ddc49a3a9dba1d891aeccc7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:08.441819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '708734ba3ddc49a3a9dba1d891aeccc7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:08.533531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '708734ba3ddc49a3a9dba1d891aeccc7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:08.544972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '708734ba3ddc49a3a9dba1d891aeccc7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:08.545860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b68f0394-d520-4156-a44d-ba3ed76b3b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '708734ba3ddc49a3a9dba1d891aeccc7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:08.570066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1288/2595] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:12:08 2022] DELETE /identity/v3/users/708734ba3ddc49a3a9dba1d891aeccc7 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:12:09.070439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:09.089191 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dbf95-53a0-4c4d-acac-02cd791fd20e service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:09.204406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:09.221265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:09.221265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:09.221265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] PATH_INFO: `/v3/projects/9dccdb0188264f2686d7275e19f39d05` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:09.221265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dccdb0188264f2686d7275e19f39d05)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:09.221265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:09.421136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dccdb0188264f2686d7275e19f39d05', 'request_id': 'req-8c2aff35-a798-4de1-8cfd-65add331321f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:09.428308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dccdb0188264f2686d7275e19f39d05', 'request_id': 'req-8c2aff35-a798-4de1-8cfd-65add331321f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:09.557626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dccdb0188264f2686d7275e19f39d05', 'request_id': 'req-8c2aff35-a798-4de1-8cfd-65add331321f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:09.573270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] The token cache is being invalidate because project 9dccdb0188264f2686d7275e19f39d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:12:09.573561 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:09.574229 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c2aff35-a798-4de1-8cfd-65add331321f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:12:09.576548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1308/2596] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:12:09 2022] DELETE /identity/v3/projects/9dccdb0188264f2686d7275e19f39d05 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:12:09.582062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4315ed-e3d3-4812-8b6b-2da18456b2bc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:09.582655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4315ed-e3d3-4812-8b6b-2da18456b2bc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:09.582773 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4315ed-e3d3-4812-8b6b-2da18456b2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:09.603795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5b4315ed-e3d3-4812-8b6b-2da18456b2bc None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:12:09.605163 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5b4315ed-e3d3-4812-8b6b-2da18456b2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:09.677363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1289/2597] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:12:09.685246 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb140bd-2b7d-4b3f-9a71-6accf5780765 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:09.685535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb140bd-2b7d-4b3f-9a71-6accf5780765 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:09.685795 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb140bd-2b7d-4b3f-9a71-6accf5780765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:09.709222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1bb140bd-2b7d-4b3f-9a71-6accf5780765 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:12:09.711119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1bb140bd-2b7d-4b3f-9a71-6accf5780765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:09.719957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1309/2598] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:12:09.726479 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4315ed-e3d3-4812-8b6b-2da18456b2bc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:09.739350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4315ed-e3d3-4812-8b6b-2da18456b2bc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:09.744518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:09.745405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:09.745692 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:09.745973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:09.746728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:12:09.747077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:09.748752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:09.767192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1290/2599] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:12:09.771007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb140bd-2b7d-4b3f-9a71-6accf5780765 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:09.783290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb140bd-2b7d-4b3f-9a71-6accf5780765 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:09.788303 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66268b18-a880-4888-acab-ab5fb271d1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:09.789084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66268b18-a880-4888-acab-ab5fb271d1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:09.789361 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66268b18-a880-4888-acab-ab5fb271d1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:09.789636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66268b18-a880-4888-acab-ab5fb271d1f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:09.790385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66268b18-a880-4888-acab-ab5fb271d1f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:09.792359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66268b18-a880-4888-acab-ab5fb271d1f0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:09.823454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:12:09.948078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1310/2600] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:12:09 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:12:09.952626 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:09.965561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39dc72-9e00-4bec-987c-a8f08bcf693f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:10.042133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87d2b2-ca9e-421a-b925-1e05c80e61e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:10.042855 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87d2b2-ca9e-421a-b925-1e05c80e61e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:10.043135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87d2b2-ca9e-421a-b925-1e05c80e61e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:10.043445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87d2b2-ca9e-421a-b925-1e05c80e61e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:10.044159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87d2b2-ca9e-421a-b925-1e05c80e61e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:10.046122 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87d2b2-ca9e-421a-b925-1e05c80e61e6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:10.055315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:12:10.209032 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1291/2601] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:12:09 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:12:10.213502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66268b18-a880-4888-acab-ab5fb271d1f0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:10.225601 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66268b18-a880-4888-acab-ab5fb271d1f0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:10.230694 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33b0c9-a00e-4ede-80ca-b7a9ce12ca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:10.231526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33b0c9-a00e-4ede-80ca-b7a9ce12ca8a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:10.231865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33b0c9-a00e-4ede-80ca-b7a9ce12ca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:10.232338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33b0c9-a00e-4ede-80ca-b7a9ce12ca8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:10.233085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33b0c9-a00e-4ede-80ca-b7a9ce12ca8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:10.234713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33b0c9-a00e-4ede-80ca-b7a9ce12ca8a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:10.246466 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1311/2602] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:12:10.257549 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87d2b2-ca9e-421a-b925-1e05c80e61e6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:10.269569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87d2b2-ca9e-421a-b925-1e05c80e61e6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:10.274938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdba93e-0525-4f67-902e-18bdd54bc516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:10.275678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba93e-0525-4f67-902e-18bdd54bc516 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:10.276151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba93e-0525-4f67-902e-18bdd54bc516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:10.276305 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba93e-0525-4f67-902e-18bdd54bc516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:10.277028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdba93e-0525-4f67-902e-18bdd54bc516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:10.278614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdba93e-0525-4f67-902e-18bdd54bc516 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:10.291883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1292/2603] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:12:10.297889 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33b0c9-a00e-4ede-80ca-b7a9ce12ca8a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:10.312903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33b0c9-a00e-4ede-80ca-b7a9ce12ca8a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:10.321667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:10.323292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:10.323835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:10.327846 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] PATH_INFO: `/v3/projects/9614129d1e864c3ea288aaac8387c5d1/users/73d76a988462431fa6bb03dca64463a4/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:10.355012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9614129d1e864c3ea288aaac8387c5d1, user_id=73d76a988462431fa6bb03dca64463a4, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:10.358086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:10.487725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1312/2604] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:12:10 2022] PUT /identity/v3/projects/9614129d1e864c3ea288aaac8387c5d1/users/73d76a988462431fa6bb03dca64463a4/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:12:10.493069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f9924-551e-4e08-993d-eaae58ecacef None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:10.493356 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f9924-551e-4e08-993d-eaae58ecacef None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:10.493664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f9924-551e-4e08-993d-eaae58ecacef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:10.513621 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c53f9924-551e-4e08-993d-eaae58ecacef None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:12:10.514946 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c53f9924-551e-4e08-993d-eaae58ecacef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:10.599523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1293/2605] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:12:10.614035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:10.616008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:10.679962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a04df-6f5d-4bc5-ac54-19a32a7317a3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:10.683979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:10.684657 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:10.684938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:10.685222 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:10.698678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:10.701543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:10.720263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1313/2606] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:12:17.039957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac744b-6c19-4f92-ae8a-36902f7fc22b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:17.039957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac744b-6c19-4f92-ae8a-36902f7fc22b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:17.039957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac744b-6c19-4f92-ae8a-36902f7fc22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:17.073729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-06ac744b-6c19-4f92-ae8a-36902f7fc22b None None] MFA Rules not processed for user `73d76a988462431fa6bb03dca64463a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:12:17.080106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-06ac744b-6c19-4f92-ae8a-36902f7fc22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:17.164774 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1294/2607] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:12:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:12:17.192661 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:12:17.194244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:17.277722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094ea45-7f4e-47c8-a45a-6794fca7afc0 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:12:17.283560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c894a-7a9d-4d08-8bc0-8e1bc6179b59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:12:17.284236 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c894a-7a9d-4d08-8bc0-8e1bc6179b59 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:17.284518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c894a-7a9d-4d08-8bc0-8e1bc6179b59 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:17.284794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c894a-7a9d-4d08-8bc0-8e1bc6179b59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:17.295354 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c894a-7a9d-4d08-8bc0-8e1bc6179b59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:12:17.303725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c894a-7a9d-4d08-8bc0-8e1bc6179b59 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:12:17.310450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1314/2608] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:12:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:12:20.283521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3e76-b49b-40e2-9421-d694cdcd8821 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:12:20.284298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3e76-b49b-40e2-9421-d694cdcd8821 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:12:20.284622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3e76-b49b-40e2-9421-d694cdcd8821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:12:20.318152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a52e3e76-b49b-40e2-9421-d694cdcd8821 None None] MFA Rules not processed for user `73d76a988462431fa6bb03dca64463a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:12:20.320460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a52e3e76-b49b-40e2-9421-d694cdcd8821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:12:20.333528 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1295/2609] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:12:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:12:27.302949 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:12:30.309446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:13:01.779111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42329fa-43a2-47ad-a27b-67b2f87fc5b4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:13:01.789638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42329fa-43a2-47ad-a27b-67b2f87fc5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:13:01.789638 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42329fa-43a2-47ad-a27b-67b2f87fc5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:13:01.799590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c42329fa-43a2-47ad-a27b-67b2f87fc5b4 None None] MFA Rules not processed for user `4ac71714265342098d2487a701a6c364`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:13:01.800850 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c42329fa-43a2-47ad-a27b-67b2f87fc5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:13:01.861308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1315/2610] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:13:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:13:11.853964 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=25 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:13:22.695580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e3e76-b49b-40e2-9421-d694cdcd8821 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:13:22.697102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a52e3e76-b49b-40e2-9421-d694cdcd8821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:13:22.759471 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e3e76-b49b-40e2-9421-d694cdcd8821 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:13:22.763384 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:13:22.764083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:13:22.764367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:13:22.764646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:13:22.766744 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:13:22.853268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:13:22.855109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:13:22.866743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1296/2611] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:13:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:13:31.439204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2f368-0e08-494f-95ad-96e7769a7eee None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:13:31.439807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2f368-0e08-494f-95ad-96e7769a7eee None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:13:31.439807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2f368-0e08-494f-95ad-96e7769a7eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:13:31.457801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-26c2f368-0e08-494f-95ad-96e7769a7eee None None] MFA Rules not processed for user `2e9707561b4a4811bd7a7d3fed4fcd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:13:31.458995 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-26c2f368-0e08-494f-95ad-96e7769a7eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:13:31.465874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1316/2612] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:13:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:13:32.861671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=48 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:13:37.344090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:13:37.346062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:13:37.438895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d149346-e59a-495d-a46d-3309d7b0dcc6 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:13:37.445055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:13:37.445829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:13:37.446179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:13:37.446532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:13:37.448542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:13:37.463206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:13:37.465043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:13:37.478647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1297/2613] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:13:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:13:41.452271 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:13:47.472447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=32 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:14:11.881667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa691047-422d-4eca-a3df-b8a09773fc1b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:14:11.884637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa691047-422d-4eca-a3df-b8a09773fc1b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:14:11.884637 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa691047-422d-4eca-a3df-b8a09773fc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:14:11.899852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fa691047-422d-4eca-a3df-b8a09773fc1b None None] MFA Rules not processed for user `4ac71714265342098d2487a701a6c364`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:14:11.901084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fa691047-422d-4eca-a3df-b8a09773fc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:14:11.908157 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1317/2614] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:14:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:14:21.894489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:14:29.910522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:14:29.921197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82d60b-0f54-400e-882e-da87217ac977 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:14:29.925047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:14:29.925683 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:14:29.925960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:14:29.926235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:14:29.928068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:14:29.941812 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:14:29.943514 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:14:29.954510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1298/2615] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:14:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:14:31.080442 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa691047-422d-4eca-a3df-b8a09773fc1b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:14:31.081872 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fa691047-422d-4eca-a3df-b8a09773fc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:14:31.095131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa691047-422d-4eca-a3df-b8a09773fc1b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:14:31.098851 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1f384-a325-422e-a7e9-508dcab4e3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:14:31.099535 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1f384-a325-422e-a7e9-508dcab4e3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:14:31.099852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1f384-a325-422e-a7e9-508dcab4e3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:14:31.100135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1f384-a325-422e-a7e9-508dcab4e3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:14:31.101917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-09b1f384-a325-422e-a7e9-508dcab4e3f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:14:31.116145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1f384-a325-422e-a7e9-508dcab4e3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:14:31.117577 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1f384-a325-422e-a7e9-508dcab4e3f0 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:14:31.129843 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1318/2616] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:14:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:14:39.949721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:14:41.124068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:12.532280 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:12.545208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f466-ca44-4b77-9703-7981d6cf75c1 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:12.551853 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:12.555288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:12.555288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:12.555288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:12.555288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:12.572270 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:12.574161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:12.586838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1299/2617] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 04:15:17.858035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709af4d1-910a-4f3c-bc8f-461daeb3566c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:17.860471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709af4d1-910a-4f3c-bc8f-461daeb3566c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:17.860471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709af4d1-910a-4f3c-bc8f-461daeb3566c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:17.877406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-709af4d1-910a-4f3c-bc8f-461daeb3566c None None] MFA Rules not processed for user `73d76a988462431fa6bb03dca64463a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:17.878632 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-709af4d1-910a-4f3c-bc8f-461daeb3566c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:17.885845 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1319/2618] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:21.393129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:21.403334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695016f5-416e-4393-b22d-e3a544648ae1 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:21.407654 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:21.408343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:21.408602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:21.408873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:21.410704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:21.420655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:21.422707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:21.431058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1300/2619] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:27.872073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:28.525761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709af4d1-910a-4f3c-bc8f-461daeb3566c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:28.527332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-709af4d1-910a-4f3c-bc8f-461daeb3566c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:28.542544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709af4d1-910a-4f3c-bc8f-461daeb3566c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:28.547320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:28.548228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:28.548514 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:28.548791 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] PATH_INFO: `/v3/users/4ac71714265342098d2487a701a6c364` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:28.550008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac71714265342098d2487a701a6c364)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:28.552061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:28.761235 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac71714265342098d2487a701a6c364'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:28.762064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac71714265342098d2487a701a6c364'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:28.762754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac71714265342098d2487a701a6c364'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:28.777548 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac71714265342098d2487a701a6c364'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:28.795747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-870d015b-8836-40a9-ada5-df5a560873bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac71714265342098d2487a701a6c364'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:28.918325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1320/2620] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:28 2022] DELETE /identity/v3/users/4ac71714265342098d2487a701a6c364 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:29.194545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:29.208154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43d40e-8e8b-4edb-84b8-814a21b6b7d3 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:29.277302 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:29.278156 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:29.278435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:29.278708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] PATH_INFO: `/v3/projects/f76a684261dd49f68a42a4e157355f37` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:29.279927 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76a684261dd49f68a42a4e157355f37)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:29.281930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:29.403501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76a684261dd49f68a42a4e157355f37', 'request_id': 'req-06544be9-4c7b-4261-8ea6-81a7f2272aab'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:29.409489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76a684261dd49f68a42a4e157355f37', 'request_id': 'req-06544be9-4c7b-4261-8ea6-81a7f2272aab'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:29.415906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76a684261dd49f68a42a4e157355f37', 'request_id': 'req-06544be9-4c7b-4261-8ea6-81a7f2272aab'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:29.500757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] The token cache is being invalidate because project f76a684261dd49f68a42a4e157355f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:29.501074 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:29.501695 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06544be9-4c7b-4261-8ea6-81a7f2272aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:29.503277 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1301/2621] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:29 2022] DELETE /identity/v3/projects/f76a684261dd49f68a42a4e157355f37 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:29.508135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a9fce-c70c-4053-9751-73515c961aa4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:29.508417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a9fce-c70c-4053-9751-73515c961aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:29.508695 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a9fce-c70c-4053-9751-73515c961aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:29.527836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-393a9fce-c70c-4053-9751-73515c961aa4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:29.529089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-393a9fce-c70c-4053-9751-73515c961aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:29.593136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1321/2622] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:29.601535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e2844-aa8b-4ce9-9f87-3f4f1a19b952 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:29.601827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e2844-aa8b-4ce9-9f87-3f4f1a19b952 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:29.602109 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e2844-aa8b-4ce9-9f87-3f4f1a19b952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:29.623575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-462e2844-aa8b-4ce9-9f87-3f4f1a19b952 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:29.624900 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-462e2844-aa8b-4ce9-9f87-3f4f1a19b952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:29.634949 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1302/2623] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:29.639209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a9fce-c70c-4053-9751-73515c961aa4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:29.649628 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a9fce-c70c-4053-9751-73515c961aa4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:29.655717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:29.656918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:29.657379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:29.657654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:29.658376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:15:29.658666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:29.660308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:29.673424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1322/2624] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:15:29.677729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e2844-aa8b-4ce9-9f87-3f4f1a19b952 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:29.688671 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e2844-aa8b-4ce9-9f87-3f4f1a19b952 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:29.694308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8338e-7c88-49a9-9693-0d2812f3cf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:29.695083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8338e-7c88-49a9-9693-0d2812f3cf25 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:29.695515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8338e-7c88-49a9-9693-0d2812f3cf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:29.695890 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8338e-7c88-49a9-9693-0d2812f3cf25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:29.696691 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8338e-7c88-49a9-9693-0d2812f3cf25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:29.698803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8338e-7c88-49a9-9693-0d2812f3cf25 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:29.788297 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1303/2625] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:15:29 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:15:29.792404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:29.803767 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d11ac-ab81-4e94-9eda-74b3785d8a49 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:29.860702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab393c8b-0946-4e35-8802-07a7f3a6f2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:29.861382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393c8b-0946-4e35-8802-07a7f3a6f2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:29.861665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393c8b-0946-4e35-8802-07a7f3a6f2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:29.861940 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393c8b-0946-4e35-8802-07a7f3a6f2e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:29.862636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab393c8b-0946-4e35-8802-07a7f3a6f2e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:29.864587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab393c8b-0946-4e35-8802-07a7f3a6f2e7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:29.871314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:29.924612 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1323/2626] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:15:29 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:29.928483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8338e-7c88-49a9-9693-0d2812f3cf25 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:29.939264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8338e-7c88-49a9-9693-0d2812f3cf25 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:29.944365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dc465-6894-411b-9647-9a1c3b39f1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:29.945063 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc465-6894-411b-9647-9a1c3b39f1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:29.945342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc465-6894-411b-9647-9a1c3b39f1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:29.945618 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc465-6894-411b-9647-9a1c3b39f1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:29.946349 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9dc465-6894-411b-9647-9a1c3b39f1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:29.947969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9dc465-6894-411b-9647-9a1c3b39f1a6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:29.959724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1304/2627] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:29.963917 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab393c8b-0946-4e35-8802-07a7f3a6f2e7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:29.975049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab393c8b-0946-4e35-8802-07a7f3a6f2e7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:29.980018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484abff0-5405-4619-a247-8d53729eaeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:29.980785 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abff0-5405-4619-a247-8d53729eaeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:29.981074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abff0-5405-4619-a247-8d53729eaeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:29.981352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abff0-5405-4619-a247-8d53729eaeb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:29.982080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484abff0-5405-4619-a247-8d53729eaeb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:29.983731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484abff0-5405-4619-a247-8d53729eaeb2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:29.995268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1324/2628] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:29.998899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dc465-6894-411b-9647-9a1c3b39f1a6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:30.009079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dc465-6894-411b-9647-9a1c3b39f1a6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:30.013915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2a4a1-d922-417b-906a-bfa9d6007d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:30.014795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2a4a1-d922-417b-906a-bfa9d6007d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.015082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2a4a1-d922-417b-906a-bfa9d6007d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.015948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2a4a1-d922-417b-906a-bfa9d6007d8c admin admin] PATH_INFO: `/v3/projects/6bee7e965d3040eeaa2c4f87b4ece512/users/9535514d82784fb48555fb87283707fa/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.030639 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2a4a1-d922-417b-906a-bfa9d6007d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bee7e965d3040eeaa2c4f87b4ece512, user_id=9535514d82784fb48555fb87283707fa, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:30.033934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2a4a1-d922-417b-906a-bfa9d6007d8c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:30.102592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:30.107510 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1305/2629] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:29 2022] PUT /identity/v3/projects/6bee7e965d3040eeaa2c4f87b4ece512/users/9535514d82784fb48555fb87283707fa/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:30.118482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8275f-086f-4579-a035-151e7615d6dd None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.127333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8275f-086f-4579-a035-151e7615d6dd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.127333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8275f-086f-4579-a035-151e7615d6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.145366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a4e8275f-086f-4579-a035-151e7615d6dd None None] MFA Rules not processed for user `9535514d82784fb48555fb87283707fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:30.146583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a4e8275f-086f-4579-a035-151e7615d6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:30.213376 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1325/2630] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:30.223553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2a4a1-d922-417b-906a-bfa9d6007d8c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:30.285705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2a4a1-d922-417b-906a-bfa9d6007d8c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:30.290396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840761f0-b054-4e2d-96b5-797e45e37c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:30.291051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840761f0-b054-4e2d-96b5-797e45e37c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.291372 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840761f0-b054-4e2d-96b5-797e45e37c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.291690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840761f0-b054-4e2d-96b5-797e45e37c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.292336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840761f0-b054-4e2d-96b5-797e45e37c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:30.294213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840761f0-b054-4e2d-96b5-797e45e37c6f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:30.364383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1306/2631] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:15:30 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:30.368488 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8275f-086f-4579-a035-151e7615d6dd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:30.380022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8275f-086f-4579-a035-151e7615d6dd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:30.437883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa8ed8-71c6-4e1f-bc00-f7fdd59674c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:30.438583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8ed8-71c6-4e1f-bc00-f7fdd59674c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.438871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8ed8-71c6-4e1f-bc00-f7fdd59674c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.439142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8ed8-71c6-4e1f-bc00-f7fdd59674c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.439860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa8ed8-71c6-4e1f-bc00-f7fdd59674c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:30.441842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa8ed8-71c6-4e1f-bc00-f7fdd59674c6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:30.512684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1326/2632] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:15:30 2022] POST /identity/v3/users => generated 297 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:30.517071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840761f0-b054-4e2d-96b5-797e45e37c6f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:30.528174 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840761f0-b054-4e2d-96b5-797e45e37c6f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:30.532914 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfdd64-63df-4a9e-9df4-a6d700eda9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:30.533579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfdd64-63df-4a9e-9df4-a6d700eda9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.533864 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfdd64-63df-4a9e-9df4-a6d700eda9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.534144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfdd64-63df-4a9e-9df4-a6d700eda9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.534874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfdd64-63df-4a9e-9df4-a6d700eda9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:30.536509 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfdd64-63df-4a9e-9df4-a6d700eda9a4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:30.548199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1307/2633] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:30.552520 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa8ed8-71c6-4e1f-bc00-f7fdd59674c6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:30.563245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa8ed8-71c6-4e1f-bc00-f7fdd59674c6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:30.568226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287565fd-b7f2-4506-aca2-c2b4d0590bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:30.569112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287565fd-b7f2-4506-aca2-c2b4d0590bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.569394 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287565fd-b7f2-4506-aca2-c2b4d0590bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.569665 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287565fd-b7f2-4506-aca2-c2b4d0590bc6 admin admin] PATH_INFO: `/v3/projects/a97716d0815146358b473deb01c2b7e8/users/a780a673573b4a888162ae2fd47e46e3/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.585427 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287565fd-b7f2-4506-aca2-c2b4d0590bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a97716d0815146358b473deb01c2b7e8, user_id=a780a673573b4a888162ae2fd47e46e3, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:30.588513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287565fd-b7f2-4506-aca2-c2b4d0590bc6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:30.690257 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1327/2634] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:30 2022] PUT /identity/v3/projects/a97716d0815146358b473deb01c2b7e8/users/a780a673573b4a888162ae2fd47e46e3/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:30.694460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfdd64-63df-4a9e-9df4-a6d700eda9a4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:30.705741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfdd64-63df-4a9e-9df4-a6d700eda9a4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:30.769679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55483b-bd05-4a36-ab26-b0f3f196da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:30.770342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55483b-bd05-4a36-ab26-b0f3f196da8b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.770622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55483b-bd05-4a36-ab26-b0f3f196da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.770901 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55483b-bd05-4a36-ab26-b0f3f196da8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.771662 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55483b-bd05-4a36-ab26-b0f3f196da8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:30.773238 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55483b-bd05-4a36-ab26-b0f3f196da8b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:30.784328 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1308/2635] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:30.788639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287565fd-b7f2-4506-aca2-c2b4d0590bc6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:30.800156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287565fd-b7f2-4506-aca2-c2b4d0590bc6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:30.805292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:30.806185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.806473 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.806890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] PATH_INFO: `/v3/projects/a97716d0815146358b473deb01c2b7e8/users/a780a673573b4a888162ae2fd47e46e3/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.808977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a97716d0815146358b473deb01c2b7e8, user_id=a780a673573b4a888162ae2fd47e46e3, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:30.811736 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:30.824810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1328/2636] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:30 2022] PUT /identity/v3/projects/a97716d0815146358b473deb01c2b7e8/users/a780a673573b4a888162ae2fd47e46e3/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:30.834397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dd55f-c824-4199-a681-68506677b010 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:30.834690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dd55f-c824-4199-a681-68506677b010 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:30.834974 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dd55f-c824-4199-a681-68506677b010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:30.865964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-237dd55f-c824-4199-a681-68506677b010 None None] MFA Rules not processed for user `a780a673573b4a888162ae2fd47e46e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:30.867218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-237dd55f-c824-4199-a681-68506677b010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:30.930965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:30.948612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1309/2637] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:30.951867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:30.953685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:30.976320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:31.012542 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737c23b-5c42-4b69-a852-3230ba513a35 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:31.016278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6acb06-db15-4ec6-a0a5-6a3e4492a9ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:31.021019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6acb06-db15-4ec6-a0a5-6a3e4492a9ea service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:31.021019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6acb06-db15-4ec6-a0a5-6a3e4492a9ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:31.021019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6acb06-db15-4ec6-a0a5-6a3e4492a9ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:31.026836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6acb06-db15-4ec6-a0a5-6a3e4492a9ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:31.028308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6acb06-db15-4ec6-a0a5-6a3e4492a9ea service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:31.038849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1329/2638] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:31.561913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dd55f-c824-4199-a681-68506677b010 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:31.629666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dd55f-c824-4199-a681-68506677b010 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:31.634405 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:31.635315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:31.635599 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:31.635875 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] PATH_INFO: `/v3/users/9535514d82784fb48555fb87283707fa` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:31.637102 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9535514d82784fb48555fb87283707fa)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:31.639051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:31.826526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9535514d82784fb48555fb87283707fa'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:31.827191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9535514d82784fb48555fb87283707fa'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:31.889275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9535514d82784fb48555fb87283707fa'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:31.899016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9535514d82784fb48555fb87283707fa'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:31.899815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9535514d82784fb48555fb87283707fa'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:31.924987 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1310/2639] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:31 2022] DELETE /identity/v3/users/9535514d82784fb48555fb87283707fa => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:31.929415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b17402-53ca-4c4f-b827-ef7dffa0cf61 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:31.929702 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b17402-53ca-4c4f-b827-ef7dffa0cf61 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:31.929976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b17402-53ca-4c4f-b827-ef7dffa0cf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:31.949004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-41b17402-53ca-4c4f-b827-ef7dffa0cf61 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:31.950263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-41b17402-53ca-4c4f-b827-ef7dffa0cf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:32.012406 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1330/2640] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:32.022989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:32.024543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:32.087426 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cc8a8-d4ec-44ed-ae57-e001097bbbae admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:32.090960 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22c085-6173-44da-9d97-605c3d558fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:32.091647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22c085-6173-44da-9d97-605c3d558fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:32.091932 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22c085-6173-44da-9d97-605c3d558fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:32.092208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22c085-6173-44da-9d97-605c3d558fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:32.101622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22c085-6173-44da-9d97-605c3d558fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:32.103060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22c085-6173-44da-9d97-605c3d558fb8 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:32.113377 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1311/2641] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:32.962610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b17402-53ca-4c4f-b827-ef7dffa0cf61 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:32.964744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-41b17402-53ca-4c4f-b827-ef7dffa0cf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:32.979649 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b17402-53ca-4c4f-b827-ef7dffa0cf61 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:32.985996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:32.986879 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:32.987169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:32.987501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] PATH_INFO: `/v3/projects/6bee7e965d3040eeaa2c4f87b4ece512` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:32.997727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bee7e965d3040eeaa2c4f87b4ece512)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:33.000598 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:33.118046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bee7e965d3040eeaa2c4f87b4ece512', 'request_id': 'req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.125993 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bee7e965d3040eeaa2c4f87b4ece512', 'request_id': 'req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.215205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bee7e965d3040eeaa2c4f87b4ece512', 'request_id': 'req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.229887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] The token cache is being invalidate because project 6bee7e965d3040eeaa2c4f87b4ece512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:33.230379 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.231893 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.233495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1331/2642] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:32 2022] DELETE /identity/v3/projects/6bee7e965d3040eeaa2c4f87b4ece512 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:33.237570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22c085-6173-44da-9d97-605c3d558fb8 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:33.248784 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22c085-6173-44da-9d97-605c3d558fb8 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:33.265341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:33.317023 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:33.318574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:33.319049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:33.319899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] PATH_INFO: `/v3/users/a780a673573b4a888162ae2fd47e46e3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:33.321331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a780a673573b4a888162ae2fd47e46e3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:33.324677 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:33.533895 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a780a673573b4a888162ae2fd47e46e3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.535410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a780a673573b4a888162ae2fd47e46e3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.558244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:33.559942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:33.621500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:33.632391 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a780a673573b4a888162ae2fd47e46e3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.642253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a780a673573b4a888162ae2fd47e46e3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.643055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a780a673573b4a888162ae2fd47e46e3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:33.650554 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b2ee-322d-48f7-a1b4-3359b7a2864f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:33.658399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8502e-32a3-45c7-b4dc-da4c04058104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:33.659141 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8502e-32a3-45c7-b4dc-da4c04058104 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:33.659539 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8502e-32a3-45c7-b4dc-da4c04058104 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:33.659887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8502e-32a3-45c7-b4dc-da4c04058104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:33.661796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ddf8502e-32a3-45c7-b4dc-da4c04058104 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:33.674567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1312/2643] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:33 2022] DELETE /identity/v3/users/a780a673573b4a888162ae2fd47e46e3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:33.681719 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8502e-32a3-45c7-b4dc-da4c04058104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:33.683634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8502e-32a3-45c7-b4dc-da4c04058104 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:33.697727 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1332/2644] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:34.353515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:34.355181 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:34.461470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ce5d8-2a87-4bd7-8ad3-aefa8fa39889 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:34.466535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:34.467469 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:34.467809 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:34.469998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] PATH_INFO: `/v3/projects/a97716d0815146358b473deb01c2b7e8` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:34.471581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a97716d0815146358b473deb01c2b7e8)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:34.476532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:34.605417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:34.614592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a97716d0815146358b473deb01c2b7e8', 'request_id': 'req-e29e4509-f6b0-4296-a933-9fcedf76cd58'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:34.622975 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a97716d0815146358b473deb01c2b7e8', 'request_id': 'req-e29e4509-f6b0-4296-a933-9fcedf76cd58'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:34.630449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a97716d0815146358b473deb01c2b7e8', 'request_id': 'req-e29e4509-f6b0-4296-a933-9fcedf76cd58'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:34.798617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] The token cache is being invalidate because project a97716d0815146358b473deb01c2b7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:34.798909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:34.799638 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e29e4509-f6b0-4296-a933-9fcedf76cd58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:34.801298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1313/2645] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:34 2022] DELETE /identity/v3/projects/a97716d0815146358b473deb01c2b7e8 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:34.807049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b6d9-2546-429b-9907-93907469385a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:34.807668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b6d9-2546-429b-9907-93907469385a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:34.807901 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b6d9-2546-429b-9907-93907469385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:34.835039 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d108b6d9-2546-429b-9907-93907469385a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:34.836667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d108b6d9-2546-429b-9907-93907469385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:34.914888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1333/2646] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:34.923070 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e6c2c-2b08-426c-9b10-e00a564a765d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:34.923429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e6c2c-2b08-426c-9b10-e00a564a765d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:34.923697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e6c2c-2b08-426c-9b10-e00a564a765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:34.947590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-926e6c2c-2b08-426c-9b10-e00a564a765d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:34.948838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-926e6c2c-2b08-426c-9b10-e00a564a765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:34.957815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1314/2647] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:34.962249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108b6d9-2546-429b-9907-93907469385a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:34.972404 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108b6d9-2546-429b-9907-93907469385a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:34.977215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:34.977868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:34.978150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:34.978423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:34.979364 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:15:34.979634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:34.981234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:34.993411 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1334/2648] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 04:15:34.998003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e6c2c-2b08-426c-9b10-e00a564a765d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.008235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e6c2c-2b08-426c-9b10-e00a564a765d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.015981 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7f559-6f95-44fd-bd7a-06e1a9bc2a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.016843 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7f559-6f95-44fd-bd7a-06e1a9bc2a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.017138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7f559-6f95-44fd-bd7a-06e1a9bc2a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.017419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7f559-6f95-44fd-bd7a-06e1a9bc2a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.018136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7f559-6f95-44fd-bd7a-06e1a9bc2a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.020450 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7f559-6f95-44fd-bd7a-06e1a9bc2a36 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:35.103114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1315/2649] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:15:34 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:35.107146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.117822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960235a-5dff-4fb7-8099-e5389bcd6c5e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.175797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3987ff6-dd52-4087-9513-f8dc42b0fdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.176481 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3987ff6-dd52-4087-9513-f8dc42b0fdca admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.176764 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3987ff6-dd52-4087-9513-f8dc42b0fdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.181711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3987ff6-dd52-4087-9513-f8dc42b0fdca admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.181711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3987ff6-dd52-4087-9513-f8dc42b0fdca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.181711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3987ff6-dd52-4087-9513-f8dc42b0fdca admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:35.261365 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1335/2650] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:15:35 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:35.266386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7f559-6f95-44fd-bd7a-06e1a9bc2a36 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.280047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7f559-6f95-44fd-bd7a-06e1a9bc2a36 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.285073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b3ee5-1fea-45c3-9c4a-75e287051f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.285789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b3ee5-1fea-45c3-9c4a-75e287051f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.286104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b3ee5-1fea-45c3-9c4a-75e287051f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.286415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b3ee5-1fea-45c3-9c4a-75e287051f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.287157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b3ee5-1fea-45c3-9c4a-75e287051f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.288790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b3ee5-1fea-45c3-9c4a-75e287051f7f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:35.301428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1316/2651] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:35.306291 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3987ff6-dd52-4087-9513-f8dc42b0fdca admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.317844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3987ff6-dd52-4087-9513-f8dc42b0fdca admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.330241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cb2ba-92f3-48ee-bf1d-53bdde779327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.331046 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cb2ba-92f3-48ee-bf1d-53bdde779327 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.331544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cb2ba-92f3-48ee-bf1d-53bdde779327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.332106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cb2ba-92f3-48ee-bf1d-53bdde779327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.333864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cb2ba-92f3-48ee-bf1d-53bdde779327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.336793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cb2ba-92f3-48ee-bf1d-53bdde779327 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:35.352859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1336/2652] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:35.354134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b3ee5-1fea-45c3-9c4a-75e287051f7f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.365367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b3ee5-1fea-45c3-9c4a-75e287051f7f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.370616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c43e07-6595-4276-8057-53b75487c5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.371545 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c43e07-6595-4276-8057-53b75487c5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.371830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c43e07-6595-4276-8057-53b75487c5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.372106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c43e07-6595-4276-8057-53b75487c5ad admin admin] PATH_INFO: `/v3/projects/1d38a807ff7b435db401808a65a2c36a/users/e69978d01a994ee790efc9e70b47fc94/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.388046 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c43e07-6595-4276-8057-53b75487c5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d38a807ff7b435db401808a65a2c36a, user_id=e69978d01a994ee790efc9e70b47fc94, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.390828 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c43e07-6595-4276-8057-53b75487c5ad admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:35.450066 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1317/2653] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:35 2022] PUT /identity/v3/projects/1d38a807ff7b435db401808a65a2c36a/users/e69978d01a994ee790efc9e70b47fc94/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:35.461715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64192c-81b1-4736-b34d-6118146fda84 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.462008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64192c-81b1-4736-b34d-6118146fda84 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.462283 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64192c-81b1-4736-b34d-6118146fda84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.490606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fd64192c-81b1-4736-b34d-6118146fda84 None None] MFA Rules not processed for user `e69978d01a994ee790efc9e70b47fc94`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:35.491860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fd64192c-81b1-4736-b34d-6118146fda84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:35.519865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:35.559292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1337/2654] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:35.563857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c43e07-6595-4276-8057-53b75487c5ad admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.630762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c43e07-6595-4276-8057-53b75487c5ad admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.635526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f25c5b-0660-42a0-ae8b-d94175512649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.636187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f25c5b-0660-42a0-ae8b-d94175512649 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.636466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f25c5b-0660-42a0-ae8b-d94175512649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.636764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f25c5b-0660-42a0-ae8b-d94175512649 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.637448 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f25c5b-0660-42a0-ae8b-d94175512649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.640262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f25c5b-0660-42a0-ae8b-d94175512649 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:35.707275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1318/2655] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:15:35 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:35.711692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64192c-81b1-4736-b34d-6118146fda84 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.723305 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64192c-81b1-4736-b34d-6118146fda84 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.778701 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834c81a-4b0d-40cc-af55-6d2be4ee5bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.779395 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834c81a-4b0d-40cc-af55-6d2be4ee5bec admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.779678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834c81a-4b0d-40cc-af55-6d2be4ee5bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.779954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834c81a-4b0d-40cc-af55-6d2be4ee5bec admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.780640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834c81a-4b0d-40cc-af55-6d2be4ee5bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.783049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834c81a-4b0d-40cc-af55-6d2be4ee5bec admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:35.857790 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1338/2656] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:15:35 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:35.859030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f25c5b-0660-42a0-ae8b-d94175512649 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.869431 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f25c5b-0660-42a0-ae8b-d94175512649 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.874142 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf661e-5d2d-4766-b043-731efa9b9032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.874811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf661e-5d2d-4766-b043-731efa9b9032 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.875092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf661e-5d2d-4766-b043-731efa9b9032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.875403 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf661e-5d2d-4766-b043-731efa9b9032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.876123 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf661e-5d2d-4766-b043-731efa9b9032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.877697 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf661e-5d2d-4766-b043-731efa9b9032 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:35.888477 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1319/2657] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:35.895800 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834c81a-4b0d-40cc-af55-6d2be4ee5bec admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:35.907315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834c81a-4b0d-40cc-af55-6d2be4ee5bec admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:35.912536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4a6d3-d310-4bd8-8b7d-92d5bfb24a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:35.913591 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4a6d3-d310-4bd8-8b7d-92d5bfb24a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:35.913740 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4a6d3-d310-4bd8-8b7d-92d5bfb24a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:35.914021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4a6d3-d310-4bd8-8b7d-92d5bfb24a23 admin admin] PATH_INFO: `/v3/projects/faec96ff9df94458b046d81a7de8c671/users/875da2593e414e3baeef8470539fcaf2/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:35.929910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4a6d3-d310-4bd8-8b7d-92d5bfb24a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faec96ff9df94458b046d81a7de8c671, user_id=875da2593e414e3baeef8470539fcaf2, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:35.932651 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4a6d3-d310-4bd8-8b7d-92d5bfb24a23 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:36.010880 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1339/2658] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:35 2022] PUT /identity/v3/projects/faec96ff9df94458b046d81a7de8c671/users/875da2593e414e3baeef8470539fcaf2/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:36.015051 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf661e-5d2d-4766-b043-731efa9b9032 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:36.025652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf661e-5d2d-4766-b043-731efa9b9032 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:36.059154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:36.082808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1212e-f648-400b-b473-1e482722c9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:36.083525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1212e-f648-400b-b473-1e482722c9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:36.083813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1212e-f648-400b-b473-1e482722c9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:36.084092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1212e-f648-400b-b473-1e482722c9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:36.084830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1212e-f648-400b-b473-1e482722c9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:36.086428 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1212e-f648-400b-b473-1e482722c9e2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:36.098016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1320/2659] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:36.101844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4a6d3-d310-4bd8-8b7d-92d5bfb24a23 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:36.112948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4a6d3-d310-4bd8-8b7d-92d5bfb24a23 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:36.117867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:36.118748 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:36.119033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:36.119349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] PATH_INFO: `/v3/projects/faec96ff9df94458b046d81a7de8c671/users/875da2593e414e3baeef8470539fcaf2/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:36.124185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faec96ff9df94458b046d81a7de8c671, user_id=875da2593e414e3baeef8470539fcaf2, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:36.124185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:36.134298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1340/2660] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:36 2022] PUT /identity/v3/projects/faec96ff9df94458b046d81a7de8c671/users/875da2593e414e3baeef8470539fcaf2/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:36.145216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d605044-be0c-430a-a9a5-213d19ed4482 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:36.146595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d605044-be0c-430a-a9a5-213d19ed4482 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:36.146595 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d605044-be0c-430a-a9a5-213d19ed4482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:36.174432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8d605044-be0c-430a-a9a5-213d19ed4482 None None] MFA Rules not processed for user `875da2593e414e3baeef8470539fcaf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:36.175694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8d605044-be0c-430a-a9a5-213d19ed4482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:36.249104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1321/2661] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:36.259920 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:36.261648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:36.320575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88a4b9-5197-4698-babb-bd7a985f5b23 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:36.324715 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfceb2c4-b697-4343-863c-3cef30a2e9c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:36.325459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceb2c4-b697-4343-863c-3cef30a2e9c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:36.325771 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceb2c4-b697-4343-863c-3cef30a2e9c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:36.326011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceb2c4-b697-4343-863c-3cef30a2e9c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:36.358919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:36.382840 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfceb2c4-b697-4343-863c-3cef30a2e9c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:36.384316 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfceb2c4-b697-4343-863c-3cef30a2e9c6 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:36.393677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1341/2662] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:39.426835 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d605044-be0c-430a-a9a5-213d19ed4482 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:39.621857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d605044-be0c-430a-a9a5-213d19ed4482 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:39.626602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:39.627519 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:39.627804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:39.628077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] PATH_INFO: `/v3/users/e69978d01a994ee790efc9e70b47fc94` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:39.629296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69978d01a994ee790efc9e70b47fc94)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:39.631423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:39.829144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69978d01a994ee790efc9e70b47fc94'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:39.829814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69978d01a994ee790efc9e70b47fc94'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:39.974650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69978d01a994ee790efc9e70b47fc94'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:39.983079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:39.984649 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69978d01a994ee790efc9e70b47fc94'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:39.986577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69978d01a994ee790efc9e70b47fc94'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:40.009876 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1322/2663] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:39 2022] DELETE /identity/v3/users/e69978d01a994ee790efc9e70b47fc94 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:40.011574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1edd6-a5d4-46c0-be43-13e6ec83b3ad None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:40.012048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1edd6-a5d4-46c0-be43-13e6ec83b3ad None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:40.012162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1edd6-a5d4-46c0-be43-13e6ec83b3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:40.032129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0db1edd6-a5d4-46c0-be43-13e6ec83b3ad None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:40.034020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0db1edd6-a5d4-46c0-be43-13e6ec83b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:40.107824 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1342/2664] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:15:40.122284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:40.123872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:40.196132 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ab1e-ff4b-4a8a-a063-a60a65f85ca4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:40.200396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6733d9-210e-4b9a-a773-2643e026bd9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:40.201235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6733d9-210e-4b9a-a773-2643e026bd9f service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:40.201679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6733d9-210e-4b9a-a773-2643e026bd9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:40.202113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6733d9-210e-4b9a-a773-2643e026bd9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:40.214429 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6733d9-210e-4b9a-a773-2643e026bd9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:40.217216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6733d9-210e-4b9a-a773-2643e026bd9f service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:40.240058 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1323/2665] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:40.662074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1edd6-a5d4-46c0-be43-13e6ec83b3ad None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:40.663726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0db1edd6-a5d4-46c0-be43-13e6ec83b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:40.677179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1edd6-a5d4-46c0-be43-13e6ec83b3ad None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:40.681937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:40.682879 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:40.683298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:40.683822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] PATH_INFO: `/v3/projects/1d38a807ff7b435db401808a65a2c36a` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:40.685104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d38a807ff7b435db401808a65a2c36a)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:40.687088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:40.812576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d38a807ff7b435db401808a65a2c36a', 'request_id': 'req-a318cd17-b5a2-4893-8ae0-5ea641ed258a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:40.819830 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d38a807ff7b435db401808a65a2c36a', 'request_id': 'req-a318cd17-b5a2-4893-8ae0-5ea641ed258a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:40.906471 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d38a807ff7b435db401808a65a2c36a', 'request_id': 'req-a318cd17-b5a2-4893-8ae0-5ea641ed258a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:40.919256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] The token cache is being invalidate because project 1d38a807ff7b435db401808a65a2c36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:40.919552 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:40.920220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:40.921779 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1343/2666] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:40 2022] DELETE /identity/v3/projects/1d38a807ff7b435db401808a65a2c36a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:40.926397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6733d9-210e-4b9a-a773-2643e026bd9f service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:40.936924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6733d9-210e-4b9a-a773-2643e026bd9f service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:41.007187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:41.008105 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:41.008398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:41.008674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] PATH_INFO: `/v3/users/875da2593e414e3baeef8470539fcaf2` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:41.009925 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=875da2593e414e3baeef8470539fcaf2)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:41.011890 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:41.171727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875da2593e414e3baeef8470539fcaf2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:41.172407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875da2593e414e3baeef8470539fcaf2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:41.248382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875da2593e414e3baeef8470539fcaf2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:41.256667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875da2593e414e3baeef8470539fcaf2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:41.257423 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cb6936ad-b845-428a-b395-f793ed7c5cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875da2593e414e3baeef8470539fcaf2'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:41.271024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:41.275443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1324/2667] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:40 2022] DELETE /identity/v3/users/875da2593e414e3baeef8470539fcaf2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:41.686329 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:41.689594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:41.721639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318cd17-b5a2-4893-8ae0-5ea641ed258a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:41.816032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:41.822939 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:41.823859 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:41.824146 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:41.824517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] PATH_INFO: `/v3/projects/faec96ff9df94458b046d81a7de8c671` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:41.825763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faec96ff9df94458b046d81a7de8c671)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:41.828183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:41.957902 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faec96ff9df94458b046d81a7de8c671', 'request_id': 'req-ec5c2885-8c42-4e64-97be-9cbce971db84'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:41.963560 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faec96ff9df94458b046d81a7de8c671', 'request_id': 'req-ec5c2885-8c42-4e64-97be-9cbce971db84'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:42.031991 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faec96ff9df94458b046d81a7de8c671', 'request_id': 'req-ec5c2885-8c42-4e64-97be-9cbce971db84'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:42.044259 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] The token cache is being invalidate because project faec96ff9df94458b046d81a7de8c671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:42.044550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:42.045187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ec5c2885-8c42-4e64-97be-9cbce971db84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:42.046766 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1344/2668] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:41 2022] DELETE /identity/v3/projects/faec96ff9df94458b046d81a7de8c671 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:42.052244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067fac-e777-450d-b3dd-39a30e9f5b7f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.052721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067fac-e777-450d-b3dd-39a30e9f5b7f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.052877 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25067fac-e777-450d-b3dd-39a30e9f5b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.071597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-25067fac-e777-450d-b3dd-39a30e9f5b7f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:42.072819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-25067fac-e777-450d-b3dd-39a30e9f5b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:42.130769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1325/2669] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:42.138459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594c0d-443b-46cd-acbd-656622e61d37 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.138749 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594c0d-443b-46cd-acbd-656622e61d37 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.139029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594c0d-443b-46cd-acbd-656622e61d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.157798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f1594c0d-443b-46cd-acbd-656622e61d37 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:42.159009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f1594c0d-443b-46cd-acbd-656622e61d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:42.166992 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1345/2670] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:42.171121 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25067fac-e777-450d-b3dd-39a30e9f5b7f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:42.181652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25067fac-e777-450d-b3dd-39a30e9f5b7f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:42.186110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:42.186753 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.187033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.187336 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.188082 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:15:42.188380 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:42.189984 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:42.202365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1326/2671] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:15:42.207078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1594c0d-443b-46cd-acbd-656622e61d37 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:42.216388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1594c0d-443b-46cd-acbd-656622e61d37 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:42.221742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b397c-fa9d-495f-a2b0-7eef1be88213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:42.222397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b397c-fa9d-495f-a2b0-7eef1be88213 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.222681 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b397c-fa9d-495f-a2b0-7eef1be88213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.222958 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b397c-fa9d-495f-a2b0-7eef1be88213 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.223723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b397c-fa9d-495f-a2b0-7eef1be88213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:42.225609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b397c-fa9d-495f-a2b0-7eef1be88213 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:42.313262 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1346/2672] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:15:42 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:15:42.313488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:42.322582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42b1d7-be33-47be-a7c3-8d5556bde342 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:42.386633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca130f0-0748-419a-9989-1dd9ee24e3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:42.387726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca130f0-0748-419a-9989-1dd9ee24e3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.388010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca130f0-0748-419a-9989-1dd9ee24e3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.388288 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca130f0-0748-419a-9989-1dd9ee24e3b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.388986 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca130f0-0748-419a-9989-1dd9ee24e3b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:42.391111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca130f0-0748-419a-9989-1dd9ee24e3b6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:42.547028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1327/2673] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:15:42 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:42.550996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b397c-fa9d-495f-a2b0-7eef1be88213 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:42.561465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b397c-fa9d-495f-a2b0-7eef1be88213 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:42.567053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de94b-3b66-4b75-9a0b-b8dd036ce1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:42.567397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de94b-3b66-4b75-9a0b-b8dd036ce1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.567527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de94b-3b66-4b75-9a0b-b8dd036ce1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.567734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de94b-3b66-4b75-9a0b-b8dd036ce1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.568757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407de94b-3b66-4b75-9a0b-b8dd036ce1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:42.570506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407de94b-3b66-4b75-9a0b-b8dd036ce1f7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:42.582126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1347/2674] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:42.586286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca130f0-0748-419a-9989-1dd9ee24e3b6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:42.596312 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca130f0-0748-419a-9989-1dd9ee24e3b6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:42.600930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a9b85-efde-4f39-86f9-cb2fc8f63359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:42.601598 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a9b85-efde-4f39-86f9-cb2fc8f63359 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.601867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a9b85-efde-4f39-86f9-cb2fc8f63359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.602136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a9b85-efde-4f39-86f9-cb2fc8f63359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.602867 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a9b85-efde-4f39-86f9-cb2fc8f63359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:42.604465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a9b85-efde-4f39-86f9-cb2fc8f63359 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:42.614957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1328/2675] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:42.622352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de94b-3b66-4b75-9a0b-b8dd036ce1f7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:42.633049 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de94b-3b66-4b75-9a0b-b8dd036ce1f7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:42.637646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0954040-e106-449f-a8e3-1c4c449c7724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:42.638518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0954040-e106-449f-a8e3-1c4c449c7724 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.638807 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0954040-e106-449f-a8e3-1c4c449c7724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.639089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0954040-e106-449f-a8e3-1c4c449c7724 admin admin] PATH_INFO: `/v3/projects/e93c47953cc140e0bce9aa940b5b0388/users/c3ec17d3b93c4b529d30c2e53f4db304/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.654251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0954040-e106-449f-a8e3-1c4c449c7724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93c47953cc140e0bce9aa940b5b0388, user_id=c3ec17d3b93c4b529d30c2e53f4db304, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:42.656941 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0954040-e106-449f-a8e3-1c4c449c7724 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:42.713372 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1348/2676] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:42 2022] PUT /identity/v3/projects/e93c47953cc140e0bce9aa940b5b0388/users/c3ec17d3b93c4b529d30c2e53f4db304/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:42.788416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd283fe9-43d9-4673-93f0-5720dd41f342 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.789446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd283fe9-43d9-4673-93f0-5720dd41f342 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.789868 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd283fe9-43d9-4673-93f0-5720dd41f342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.820717 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dd283fe9-43d9-4673-93f0-5720dd41f342 None None] MFA Rules not processed for user `c3ec17d3b93c4b529d30c2e53f4db304`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:42.822085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dd283fe9-43d9-4673-93f0-5720dd41f342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:42.892343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:42.901149 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1329/2677] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:42.905703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0954040-e106-449f-a8e3-1c4c449c7724 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:42.970136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0954040-e106-449f-a8e3-1c4c449c7724 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:42.975057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5e02c-d330-4e54-b721-3e5dcb7dfb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:42.975786 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5e02c-d330-4e54-b721-3e5dcb7dfb27 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:42.976069 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5e02c-d330-4e54-b721-3e5dcb7dfb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:42.976346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5e02c-d330-4e54-b721-3e5dcb7dfb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:42.977044 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5e02c-d330-4e54-b721-3e5dcb7dfb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:42.978969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5e02c-d330-4e54-b721-3e5dcb7dfb27 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:43.063988 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1349/2678] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:15:42 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:43.068322 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd283fe9-43d9-4673-93f0-5720dd41f342 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:43.080715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd283fe9-43d9-4673-93f0-5720dd41f342 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:43.141141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e4e17-01e7-4a00-ba22-97d8bf1cad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:43.141840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e4e17-01e7-4a00-ba22-97d8bf1cad81 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.142126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e4e17-01e7-4a00-ba22-97d8bf1cad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.142402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e4e17-01e7-4a00-ba22-97d8bf1cad81 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.143094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e4e17-01e7-4a00-ba22-97d8bf1cad81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:43.145110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e4e17-01e7-4a00-ba22-97d8bf1cad81 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:43.232273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1330/2679] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:15:43 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:43.237171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5e02c-d330-4e54-b721-3e5dcb7dfb27 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:43.247847 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5e02c-d330-4e54-b721-3e5dcb7dfb27 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:43.253114 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2b6b2-1ee6-4def-8760-79f1e36c4504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:43.255713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2b6b2-1ee6-4def-8760-79f1e36c4504 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.255713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2b6b2-1ee6-4def-8760-79f1e36c4504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.255713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2b6b2-1ee6-4def-8760-79f1e36c4504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.255713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2b6b2-1ee6-4def-8760-79f1e36c4504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:43.256809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2b6b2-1ee6-4def-8760-79f1e36c4504 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:43.258982 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:43.268059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1350/2680] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:43.274973 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e4e17-01e7-4a00-ba22-97d8bf1cad81 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:43.282198 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e4e17-01e7-4a00-ba22-97d8bf1cad81 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:43.286873 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719acd51-fe96-4a99-9d8c-54ca1152895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:43.287560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719acd51-fe96-4a99-9d8c-54ca1152895a admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.287846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719acd51-fe96-4a99-9d8c-54ca1152895a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.288131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719acd51-fe96-4a99-9d8c-54ca1152895a admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.288859 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719acd51-fe96-4a99-9d8c-54ca1152895a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:43.290433 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719acd51-fe96-4a99-9d8c-54ca1152895a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:43.301342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1331/2681] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:43.306531 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2b6b2-1ee6-4def-8760-79f1e36c4504 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:43.317264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2b6b2-1ee6-4def-8760-79f1e36c4504 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:43.322269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018603b6-70f8-41bb-8d88-c9eb3567e083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:43.323164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018603b6-70f8-41bb-8d88-c9eb3567e083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.323480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018603b6-70f8-41bb-8d88-c9eb3567e083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.323759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018603b6-70f8-41bb-8d88-c9eb3567e083 admin admin] PATH_INFO: `/v3/projects/ed191552e21649a089a0095bdccca7f0/users/cefc0f5ed89e4d86b471fac06e2f89f1/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.339513 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018603b6-70f8-41bb-8d88-c9eb3567e083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed191552e21649a089a0095bdccca7f0, user_id=cefc0f5ed89e4d86b471fac06e2f89f1, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:43.342127 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018603b6-70f8-41bb-8d88-c9eb3567e083 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:43.437971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1351/2682] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:43 2022] PUT /identity/v3/projects/ed191552e21649a089a0095bdccca7f0/users/cefc0f5ed89e4d86b471fac06e2f89f1/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:43.448385 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e426d-d27a-49ba-b35f-00757c591e15 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.448668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e426d-d27a-49ba-b35f-00757c591e15 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.448948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e426d-d27a-49ba-b35f-00757c591e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.477210 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ee3e426d-d27a-49ba-b35f-00757c591e15 None None] MFA Rules not processed for user `cefc0f5ed89e4d86b471fac06e2f89f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:43.478437 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ee3e426d-d27a-49ba-b35f-00757c591e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:43.544107 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1332/2683] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:43.546194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018603b6-70f8-41bb-8d88-c9eb3567e083 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:43.608895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018603b6-70f8-41bb-8d88-c9eb3567e083 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:43.614104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f3726-df2a-4c2e-b8a4-8aeb09c21737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:43.614759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3726-df2a-4c2e-b8a4-8aeb09c21737 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.615045 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3726-df2a-4c2e-b8a4-8aeb09c21737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.615359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3726-df2a-4c2e-b8a4-8aeb09c21737 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.616048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f3726-df2a-4c2e-b8a4-8aeb09c21737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:43.617973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f3726-df2a-4c2e-b8a4-8aeb09c21737 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:43.711197 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1352/2684] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:15:43 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:43.712313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e426d-d27a-49ba-b35f-00757c591e15 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:43.723806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e426d-d27a-49ba-b35f-00757c591e15 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:43.782752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305b62-590b-4440-9f2e-6c1552a482b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:43.783512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305b62-590b-4440-9f2e-6c1552a482b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.783796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305b62-590b-4440-9f2e-6c1552a482b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.784073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305b62-590b-4440-9f2e-6c1552a482b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.784778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305b62-590b-4440-9f2e-6c1552a482b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:43.786684 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305b62-590b-4440-9f2e-6c1552a482b5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:43.795676 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:43.915759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1333/2685] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:15:43 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:15:43.921238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f3726-df2a-4c2e-b8a4-8aeb09c21737 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:43.929021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f3726-df2a-4c2e-b8a4-8aeb09c21737 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:43.933875 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a711f-4800-432e-bf20-1d4cf0570a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:43.934563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a711f-4800-432e-bf20-1d4cf0570a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.934829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a711f-4800-432e-bf20-1d4cf0570a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.935105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a711f-4800-432e-bf20-1d4cf0570a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.935868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a711f-4800-432e-bf20-1d4cf0570a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:43.937498 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a711f-4800-432e-bf20-1d4cf0570a87 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:43.949058 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1353/2686] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:43.953191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305b62-590b-4440-9f2e-6c1552a482b5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:43.963994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305b62-590b-4440-9f2e-6c1552a482b5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:43.968829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce6833-a123-4501-80ad-1cb579655dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:43.970446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce6833-a123-4501-80ad-1cb579655dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:43.970813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce6833-a123-4501-80ad-1cb579655dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:43.971161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce6833-a123-4501-80ad-1cb579655dda admin admin] PATH_INFO: `/v3/projects/c0556605614f4553b450b30e31d5a137/users/ec414be98db04bdcbafd3f5c05aa0bbb/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:43.987035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce6833-a123-4501-80ad-1cb579655dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0556605614f4553b450b30e31d5a137, user_id=ec414be98db04bdcbafd3f5c05aa0bbb, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:43.989704 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce6833-a123-4501-80ad-1cb579655dda admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:44.078108 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1334/2687] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:43 2022] PUT /identity/v3/projects/c0556605614f4553b450b30e31d5a137/users/ec414be98db04bdcbafd3f5c05aa0bbb/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:44.082698 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a711f-4800-432e-bf20-1d4cf0570a87 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:44.092896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a711f-4800-432e-bf20-1d4cf0570a87 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:44.148306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421208a-5e54-40ab-9366-3a8f107801ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:44.148886 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421208a-5e54-40ab-9366-3a8f107801ed admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:44.149166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421208a-5e54-40ab-9366-3a8f107801ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:44.149438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421208a-5e54-40ab-9366-3a8f107801ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:44.150164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421208a-5e54-40ab-9366-3a8f107801ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:44.151798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421208a-5e54-40ab-9366-3a8f107801ed admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:44.161534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1354/2688] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:15:44.165778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce6833-a123-4501-80ad-1cb579655dda admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:44.176067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce6833-a123-4501-80ad-1cb579655dda admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:44.180679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:44.181550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:44.181834 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:44.182117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] PATH_INFO: `/v3/projects/c0556605614f4553b450b30e31d5a137/users/ec414be98db04bdcbafd3f5c05aa0bbb/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:44.184115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0556605614f4553b450b30e31d5a137, user_id=ec414be98db04bdcbafd3f5c05aa0bbb, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:44.186710 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:44.198264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1335/2689] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:44 2022] PUT /identity/v3/projects/c0556605614f4553b450b30e31d5a137/users/ec414be98db04bdcbafd3f5c05aa0bbb/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:44.210104 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dfa1f-0c7c-4a24-847d-259b7fbeb987 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:44.210397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dfa1f-0c7c-4a24-847d-259b7fbeb987 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:44.210679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dfa1f-0c7c-4a24-847d-259b7fbeb987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:44.238302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9c4dfa1f-0c7c-4a24-847d-259b7fbeb987 None None] MFA Rules not processed for user `ec414be98db04bdcbafd3f5c05aa0bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:44.239571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9c4dfa1f-0c7c-4a24-847d-259b7fbeb987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:44.273438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:44.309278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1355/2690] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:44.321522 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:44.323212 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:44.380556 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526a0ed-26bc-45cf-ac13-a2655c89ccf1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:44.384179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:44.384817 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:44.385096 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:44.385382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:44.442030 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:44.443666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:44.452441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1336/2691] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:46.284198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dfa1f-0c7c-4a24-847d-259b7fbeb987 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:46.284198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9c4dfa1f-0c7c-4a24-847d-259b7fbeb987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:46.292691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dfa1f-0c7c-4a24-847d-259b7fbeb987 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:46.296518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad278d-72d7-497f-aa63-ac146a0e2d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:46.297163 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad278d-72d7-497f-aa63-ac146a0e2d5e service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:46.297469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad278d-72d7-497f-aa63-ac146a0e2d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:46.297742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad278d-72d7-497f-aa63-ac146a0e2d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:46.360117 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad278d-72d7-497f-aa63-ac146a0e2d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:46.362194 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad278d-72d7-497f-aa63-ac146a0e2d5e service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:46.371383 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1356/2692] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:47.228930 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:47.230499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:47.290052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe652c2-44be-4a5b-83b5-1f94555c1678 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:47.293728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:47.294371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:47.294655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:47.294940 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:47.296814 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:47.322869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:47.359923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:47.361345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:47.373148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1337/2693] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:47.708189 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad278d-72d7-497f-aa63-ac146a0e2d5e service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:47.717422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad278d-72d7-497f-aa63-ac146a0e2d5e service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:47.721861 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7cd5e-a9be-49eb-bb2e-7b542623e579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:47.722506 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7cd5e-a9be-49eb-bb2e-7b542623e579 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:47.722803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7cd5e-a9be-49eb-bb2e-7b542623e579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:47.723064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7cd5e-a9be-49eb-bb2e-7b542623e579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:47.732707 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7cd5e-a9be-49eb-bb2e-7b542623e579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:47.734145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7cd5e-a9be-49eb-bb2e-7b542623e579 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:47.744359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1357/2694] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:51.081928 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:51.083526 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:51.162611 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f33dc1-acdb-4e70-8da3-152a63df4f2a service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:51.168072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599548d-6622-454a-8f06-6ca847368718 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:51.168967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599548d-6622-454a-8f06-6ca847368718 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:51.169249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599548d-6622-454a-8f06-6ca847368718 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:51.169523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599548d-6622-454a-8f06-6ca847368718 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:51.177645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7599548d-6622-454a-8f06-6ca847368718 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:51.184609 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599548d-6622-454a-8f06-6ca847368718 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:51.186048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599548d-6622-454a-8f06-6ca847368718 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:51.201165 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1338/2695] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:52.569351 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7cd5e-a9be-49eb-bb2e-7b542623e579 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:52.580839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7cd5e-a9be-49eb-bb2e-7b542623e579 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:52.586275 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:52.587320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:52.587607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:52.587883 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] PATH_INFO: `/v3/users/c3ec17d3b93c4b529d30c2e53f4db304` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:52.589253 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3ec17d3b93c4b529d30c2e53f4db304)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:52.591540 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:52.806033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3ec17d3b93c4b529d30c2e53f4db304'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:52.806703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3ec17d3b93c4b529d30c2e53f4db304'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:52.807521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3ec17d3b93c4b529d30c2e53f4db304'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:52.821317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3ec17d3b93c4b529d30c2e53f4db304'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:52.828586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3ec17d3b93c4b529d30c2e53f4db304'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:52.896136 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1358/2696] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:52 2022] DELETE /identity/v3/users/c3ec17d3b93c4b529d30c2e53f4db304 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:52.900725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1481e5b-66d1-4aa1-9150-f26b4702146f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:52.901560 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1481e5b-66d1-4aa1-9150-f26b4702146f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:52.902665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1481e5b-66d1-4aa1-9150-f26b4702146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:52.921879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a1481e5b-66d1-4aa1-9150-f26b4702146f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:52.923664 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a1481e5b-66d1-4aa1-9150-f26b4702146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:52.990307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1339/2697] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:53.000519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:53.002891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:53.037401 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:53.071622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bf34e-e0bf-4ee7-bada-d280e397a6ad admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:53.076018 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ce9a-7e12-4eca-80ae-8bb7452eb41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:53.076534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ce9a-7e12-4eca-80ae-8bb7452eb41d service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:53.076819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ce9a-7e12-4eca-80ae-8bb7452eb41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:53.077094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ce9a-7e12-4eca-80ae-8bb7452eb41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:53.087475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231ce9a-7e12-4eca-80ae-8bb7452eb41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:53.088909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231ce9a-7e12-4eca-80ae-8bb7452eb41d service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:53.100438 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1359/2698] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:53.479218 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1481e5b-66d1-4aa1-9150-f26b4702146f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:53.480921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a1481e5b-66d1-4aa1-9150-f26b4702146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:53.494768 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1481e5b-66d1-4aa1-9150-f26b4702146f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:53.499929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:53.501129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:53.501878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:53.502251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] PATH_INFO: `/v3/projects/e93c47953cc140e0bce9aa940b5b0388` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:53.504072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93c47953cc140e0bce9aa940b5b0388)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:53.506156 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:53.665411 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:53.668114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93c47953cc140e0bce9aa940b5b0388', 'request_id': 'req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:53.674592 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93c47953cc140e0bce9aa940b5b0388', 'request_id': 'req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:53.681180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93c47953cc140e0bce9aa940b5b0388', 'request_id': 'req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:53.747367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] The token cache is being invalidate because project e93c47953cc140e0bce9aa940b5b0388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:53.747661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:53.748412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:53.750095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1340/2699] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:53 2022] DELETE /identity/v3/projects/e93c47953cc140e0bce9aa940b5b0388 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:53.754819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ce9a-7e12-4eca-80ae-8bb7452eb41d service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:53.766286 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ce9a-7e12-4eca-80ae-8bb7452eb41d service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:53.831789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:53.832352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:53.832553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:53.832768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] PATH_INFO: `/v3/users/cefc0f5ed89e4d86b471fac06e2f89f1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:53.833996 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cefc0f5ed89e4d86b471fac06e2f89f1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:53.838495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:54.015179 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cefc0f5ed89e4d86b471fac06e2f89f1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:54.016312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cefc0f5ed89e4d86b471fac06e2f89f1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:54.017327 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cefc0f5ed89e4d86b471fac06e2f89f1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:54.038952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cefc0f5ed89e4d86b471fac06e2f89f1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:54.050496 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cefc0f5ed89e4d86b471fac06e2f89f1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:54.168405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1360/2700] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:53 2022] DELETE /identity/v3/users/cefc0f5ed89e4d86b471fac06e2f89f1 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:54.703193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:54.717067 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:54.732942 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d74fa-6b6c-4ea7-8877-6ceb7eea7ebd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:54.803193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:54.804110 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:54.804394 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:54.804668 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] PATH_INFO: `/v3/projects/ed191552e21649a089a0095bdccca7f0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:54.805998 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed191552e21649a089a0095bdccca7f0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:54.807850 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:54.969652 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed191552e21649a089a0095bdccca7f0', 'request_id': 'req-2c63c36d-71ec-48b2-ab61-13d667e6660b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:54.975318 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed191552e21649a089a0095bdccca7f0', 'request_id': 'req-2c63c36d-71ec-48b2-ab61-13d667e6660b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:54.981851 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed191552e21649a089a0095bdccca7f0', 'request_id': 'req-2c63c36d-71ec-48b2-ab61-13d667e6660b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.022314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:55.023906 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:55.048732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505f075-9419-4683-8310-8e37d6a2ca60 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:55.082708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] The token cache is being invalidate because project ed191552e21649a089a0095bdccca7f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:55.083159 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.084201 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:55.084330 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.089028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1341/2701] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:54 2022] DELETE /identity/v3/projects/ed191552e21649a089a0095bdccca7f0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:55.089862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:55.091217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:55.096836 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:55.097716 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:55.097994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:55.098270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] PATH_INFO: `/v3/users/2e9707561b4a4811bd7a7d3fed4fcd6b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:55.099679 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9707561b4a4811bd7a7d3fed4fcd6b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:55.101582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:55.156323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c36d-71ec-48b2-ab61-13d667e6660b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:55.161069 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:55.161972 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:55.162242 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:55.162517 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] PATH_INFO: `/v3/users/ec414be98db04bdcbafd3f5c05aa0bbb` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:55.163899 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec414be98db04bdcbafd3f5c05aa0bbb)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:55.165794 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:55.170585 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:55.279655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9707561b4a4811bd7a7d3fed4fcd6b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.280356 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9707561b4a4811bd7a7d3fed4fcd6b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.280990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9707561b4a4811bd7a7d3fed4fcd6b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.293603 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9707561b4a4811bd7a7d3fed4fcd6b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.301084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9707561b4a4811bd7a7d3fed4fcd6b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.324747 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec414be98db04bdcbafd3f5c05aa0bbb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.325416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec414be98db04bdcbafd3f5c05aa0bbb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.352002 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1361/2702] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:55 2022] DELETE /identity/v3/users/2e9707561b4a4811bd7a7d3fed4fcd6b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:15:55.391614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec414be98db04bdcbafd3f5c05aa0bbb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.399874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec414be98db04bdcbafd3f5c05aa0bbb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.400590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec414be98db04bdcbafd3f5c05aa0bbb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.419727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1342/2703] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:55 2022] DELETE /identity/v3/users/ec414be98db04bdcbafd3f5c05aa0bbb => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:55.588549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:55.588549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:55.663353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869e9b-8266-4c0c-bc72-c0f3757e4d0f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:55.724074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:55.724935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:55.725219 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:55.725487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] PATH_INFO: `/v3/projects/da99a87350634e9ea8a89f314f5e6118` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:55.726656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da99a87350634e9ea8a89f314f5e6118)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:55.728659 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:55.745747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:55.862138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:55.863709 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:55.878478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da99a87350634e9ea8a89f314f5e6118', 'request_id': 'req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.878698 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417540e-0d15-43e8-905d-c40867b7fcb7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:55.884362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da99a87350634e9ea8a89f314f5e6118', 'request_id': 'req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.941036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:55.941909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:55.942188 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:55.942462 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] PATH_INFO: `/v3/projects/c0556605614f4553b450b30e31d5a137` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:55.944035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0556605614f4553b450b30e31d5a137)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:55.945957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:55.969125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da99a87350634e9ea8a89f314f5e6118', 'request_id': 'req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.982686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] The token cache is being invalidate because project da99a87350634e9ea8a89f314f5e6118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:55.982999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.983666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:55.985405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1362/2704] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:55 2022] DELETE /identity/v3/projects/da99a87350634e9ea8a89f314f5e6118 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:55.989315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:55.990742 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:56.004059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3ebc-27cd-4f1b-aa2b-8506f63fb1d3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:56.009344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:56.010224 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.010566 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.010874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] PATH_INFO: `/v3/users/4966505aa6de4ef1839eeab044dc928f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.012126 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4966505aa6de4ef1839eeab044dc928f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:56.014240 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:56.067579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0556605614f4553b450b30e31d5a137', 'request_id': 'req-b3174970-6f45-4232-a69d-237ed3e45b53'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.073496 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0556605614f4553b450b30e31d5a137', 'request_id': 'req-b3174970-6f45-4232-a69d-237ed3e45b53'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.079804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0556605614f4553b450b30e31d5a137', 'request_id': 'req-b3174970-6f45-4232-a69d-237ed3e45b53'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.168145 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] The token cache is being invalidate because project c0556605614f4553b450b30e31d5a137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:56.168360 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.168969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b3174970-6f45-4232-a69d-237ed3e45b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.170501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1343/2705] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:55 2022] DELETE /identity/v3/projects/c0556605614f4553b450b30e31d5a137 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:15:56.175152 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d7770-2601-4caa-9011-145c7993121f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.175475 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d7770-2601-4caa-9011-145c7993121f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.175759 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d7770-2601-4caa-9011-145c7993121f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.188231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4966505aa6de4ef1839eeab044dc928f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.189182 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4966505aa6de4ef1839eeab044dc928f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.189804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4966505aa6de4ef1839eeab044dc928f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.195052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-aa0d7770-2601-4caa-9011-145c7993121f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:56.196323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aa0d7770-2601-4caa-9011-145c7993121f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:56.204276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4966505aa6de4ef1839eeab044dc928f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.205116 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:56.213815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4966505aa6de4ef1839eeab044dc928f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.260778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1344/2706] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:15:56.269077 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a9d0-261e-46a8-8119-05a4e51bfdb0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.269363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a9d0-261e-46a8-8119-05a4e51bfdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.269699 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a9d0-261e-46a8-8119-05a4e51bfdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.288245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b6f4a9d0-261e-46a8-8119-05a4e51bfdb0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:56.288983 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1363/2707] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:15:55 2022] DELETE /identity/v3/users/4966505aa6de4ef1839eeab044dc928f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:15:56.289474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b6f4a9d0-261e-46a8-8119-05a4e51bfdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:56.352245 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1345/2708] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:15:56.360571 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:56.371148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03b269-a327-44dd-836a-700315f5fe08 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:56.376129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:56.376885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.377264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.378118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.378927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:15:56.379349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:56.381203 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:56.395536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1364/2709] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:15:56.399918 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4a9d0-261e-46a8-8119-05a4e51bfdb0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:56.410182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4a9d0-261e-46a8-8119-05a4e51bfdb0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:56.415128 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:56.415815 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.416097 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.416369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.417071 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:56.419293 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:56.525544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1346/2710] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:15:56 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:15:56.529979 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:56.540352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46599e3d-f96a-4d49-a04b-c6d71b69de18 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:56.549134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:56.552555 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:56.605639 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281132fe-4022-4655-a4ad-3ffb9fbf7ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:56.606322 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281132fe-4022-4655-a4ad-3ffb9fbf7ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.606607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281132fe-4022-4655-a4ad-3ffb9fbf7ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.606919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281132fe-4022-4655-a4ad-3ffb9fbf7ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.607685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281132fe-4022-4655-a4ad-3ffb9fbf7ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:56.609622 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281132fe-4022-4655-a4ad-3ffb9fbf7ff2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:56.621188 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:56.633530 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b262721-a95a-49ee-b781-d1acc508f07c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:56.638164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:56.639055 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.639371 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.639714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] PATH_INFO: `/v3/projects/2977fbb8fa004c1ba5f9572e2b043c28` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.640889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2977fbb8fa004c1ba5f9572e2b043c28)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:56.642778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:56.735148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1365/2711] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:15:56 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:15:56.739525 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:15:56.739923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281132fe-4022-4655-a4ad-3ffb9fbf7ff2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:56.748086 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2977fbb8fa004c1ba5f9572e2b043c28', 'request_id': 'req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.750600 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281132fe-4022-4655-a4ad-3ffb9fbf7ff2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:56.754498 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2977fbb8fa004c1ba5f9572e2b043c28', 'request_id': 'req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.768234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2977fbb8fa004c1ba5f9572e2b043c28', 'request_id': 'req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.807244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28460-bcf4-4956-86ae-8483a7aa93d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:56.807919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28460-bcf4-4956-86ae-8483a7aa93d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.808202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28460-bcf4-4956-86ae-8483a7aa93d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.808479 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28460-bcf4-4956-86ae-8483a7aa93d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.809204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28460-bcf4-4956-86ae-8483a7aa93d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:56.810789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28460-bcf4-4956-86ae-8483a7aa93d1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:56.820734 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1366/2712] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:15:56.824640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28460-bcf4-4956-86ae-8483a7aa93d1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:56.831583 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] The token cache is being invalidate because project 2977fbb8fa004c1ba5f9572e2b043c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:15:56.831874 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.832513 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:15:56.834112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1347/2713] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:15:56 2022] DELETE /identity/v3/projects/2977fbb8fa004c1ba5f9572e2b043c28 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:15:56.834453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28460-bcf4-4956-86ae-8483a7aa93d1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:56.839174 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fbc65-f4d5-4f1c-94fc-f4634b31c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:56.857697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbc65-f4d5-4f1c-94fc-f4634b31c174 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.857697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbc65-f4d5-4f1c-94fc-f4634b31c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.857697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbc65-f4d5-4f1c-94fc-f4634b31c174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.857697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fbc65-f4d5-4f1c-94fc-f4634b31c174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:56.857697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fbc65-f4d5-4f1c-94fc-f4634b31c174 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:56.857697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1367/2714] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:15:56.859983 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:56.861532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:56.876740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d3321-ef6a-44aa-9fb6-6c1ae795cdf9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:56.882135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:56.883342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.883629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.883904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] PATH_INFO: `/v3/projects/a9f7e5a8b43740ab8cd82b939a8af457/users/90fc6c2d12f3420eb7d646d7fceb8965/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.901258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f7e5a8b43740ab8cd82b939a8af457, user_id=90fc6c2d12f3420eb7d646d7fceb8965, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:56.904006 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:56.952543 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1348/2715] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:15:56 2022] PUT /identity/v3/projects/a9f7e5a8b43740ab8cd82b939a8af457/users/90fc6c2d12f3420eb7d646d7fceb8965/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:15:56.958324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38609b-d7d2-4b2e-838c-f7bda6411bf3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:56.958615 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38609b-d7d2-4b2e-838c-f7bda6411bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:56.958894 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38609b-d7d2-4b2e-838c-f7bda6411bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:56.978756 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8b38609b-d7d2-4b2e-838c-f7bda6411bf3 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:56.979942 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8b38609b-d7d2-4b2e-838c-f7bda6411bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:57.050652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1368/2716] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:57.060748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:57.062707 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:57.126038 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d70f3e-8655-4832-bcbc-5dd198ebf5b8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:57.129901 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:57.130541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:57.130820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:57.131091 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:57.140125 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:57.141531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:57.151601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1349/2717] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:15:58.272134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba227ba-f95a-4751-9dc8-c6bf66e620be None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:58.272690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba227ba-f95a-4751-9dc8-c6bf66e620be None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:58.272690 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba227ba-f95a-4751-9dc8-c6bf66e620be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:58.301220 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9ba227ba-f95a-4751-9dc8-c6bf66e620be None None] MFA Rules not processed for user `90fc6c2d12f3420eb7d646d7fceb8965`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:15:58.302455 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9ba227ba-f95a-4751-9dc8-c6bf66e620be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:58.372332 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1369/2718] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:15:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:15:58.383567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:15:58.385187 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:15:58.445724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6e49-7da0-4dfb-a141-d919d5b8977e service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:15:58.449365 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:15:58.450003 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:15:58.450284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:15:58.450559 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:15:58.458922 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:15:58.460379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:15:58.469196 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1350/2719] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:15:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:16:00.100543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e29bf-7813-4484-8483-1c8c795ee999 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:00.101131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e29bf-7813-4484-8483-1c8c795ee999 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:00.101131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e29bf-7813-4484-8483-1c8c795ee999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:00.119293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d80e29bf-7813-4484-8483-1c8c795ee999 None None] MFA Rules not processed for user `90fc6c2d12f3420eb7d646d7fceb8965`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:00.120536 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d80e29bf-7813-4484-8483-1c8c795ee999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:00.127528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1370/2720] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:16:00.364441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:00.365840 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:00.377645 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda62d3-4332-4985-8b2e-8c5836821dd4 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:00.381400 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:00.382062 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:00.382345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:00.382623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:00.384376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:00.394341 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:00.395778 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:00.403725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1351/2721] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:16:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:16:00.781267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e29bf-7813-4484-8483-1c8c795ee999 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:00.782682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d80e29bf-7813-4484-8483-1c8c795ee999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:00.787887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:00.862910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e29bf-7813-4484-8483-1c8c795ee999 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:00.867961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9ca44-3fef-4aa2-9c88-95f73bab98b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:00.868611 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9ca44-3fef-4aa2-9c88-95f73bab98b3 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:00.868895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9ca44-3fef-4aa2-9c88-95f73bab98b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:00.869171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9ca44-3fef-4aa2-9c88-95f73bab98b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:00.870973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-97d9ca44-3fef-4aa2-9c88-95f73bab98b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:00.883012 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9ca44-3fef-4aa2-9c88-95f73bab98b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:00.884737 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9ca44-3fef-4aa2-9c88-95f73bab98b3 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:00.895549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1371/2722] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:16:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:16:10.400234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=5,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:10.975378 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=31 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:10.978939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:10.980655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:11.105880 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a3ad0-75d5-43cb-8a38-e9f0a7016964 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:11.111603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:11.112258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:11.112544 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:11.112827 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:11.114647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:11.127617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:11.129219 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:11.141018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1352/2723] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:16:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:16:18.237613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcbd09-de96-410f-b990-3ae66bf32ced None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:18.238129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcbd09-de96-410f-b990-3ae66bf32ced None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:18.238234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcbd09-de96-410f-b990-3ae66bf32ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:18.257008 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5dbcbd09-de96-410f-b990-3ae66bf32ced None None] MFA Rules not processed for user `90fc6c2d12f3420eb7d646d7fceb8965`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:18.258225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5dbcbd09-de96-410f-b990-3ae66bf32ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:18.265154 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1372/2724] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:16:18.498642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:18.517796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac6bd3-7c10-43b5-855e-6d5ac0130cfc service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:18.521463 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:18.522117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:18.522396 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:18.522672 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:18.524466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:18.549282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:18.550990 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:18.560364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1353/2725] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:16:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:16:28.250977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:28.556538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=71 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:40.455183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd36399-9dd8-498a-b0cd-0bb9c812a870 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:40.455666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd36399-9dd8-498a-b0cd-0bb9c812a870 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:40.455797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd36399-9dd8-498a-b0cd-0bb9c812a870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:40.484313 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8cd36399-9dd8-498a-b0cd-0bb9c812a870 None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:40.485526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8cd36399-9dd8-498a-b0cd-0bb9c812a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:40.549040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1373/2726] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 1 04:16:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:16:40.676424 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:40.676856 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:40.749862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8847ea3-294f-4916-a624-1ea11271119c service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:40.754870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc89d4-9dbc-4ef0-aa13-f03e5a461f49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:40.755549 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc89d4-9dbc-4ef0-aa13-f03e5a461f49 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:40.755836 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc89d4-9dbc-4ef0-aa13-f03e5a461f49 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:40.756113 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc89d4-9dbc-4ef0-aa13-f03e5a461f49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:40.764733 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc89d4-9dbc-4ef0-aa13-f03e5a461f49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:40.766304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc89d4-9dbc-4ef0-aa13-f03e5a461f49 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:40.774265 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1354/2727] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:16:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 04:16:48.267613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd36399-9dd8-498a-b0cd-0bb9c812a870 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:48.269969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8cd36399-9dd8-498a-b0cd-0bb9c812a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:48.344921 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd36399-9dd8-498a-b0cd-0bb9c812a870 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:48.350256 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:48.352247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:48.352247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:48.352247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] PATH_INFO: `/v3/users/73d76a988462431fa6bb03dca64463a4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:48.354161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d76a988462431fa6bb03dca64463a4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:48.356211 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:48.580233 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d76a988462431fa6bb03dca64463a4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:48.580994 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d76a988462431fa6bb03dca64463a4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:48.581676 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d76a988462431fa6bb03dca64463a4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:48.599153 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d76a988462431fa6bb03dca64463a4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:48.610093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-adc5f4a2-53c1-4e6a-9c31-ed0e919b7553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d76a988462431fa6bb03dca64463a4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:48.700895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1374/2728] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:16:48 2022] DELETE /identity/v3/users/73d76a988462431fa6bb03dca64463a4 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:16:49.006282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc89d4-9dbc-4ef0-aa13-f03e5a461f49 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:49.042327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc89d4-9dbc-4ef0-aa13-f03e5a461f49 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:49.116949 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:49.118251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:49.118705 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:49.118985 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] PATH_INFO: `/v3/projects/9614129d1e864c3ea288aaac8387c5d1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:49.120253 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9614129d1e864c3ea288aaac8387c5d1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:49.122161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:49.256257 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9614129d1e864c3ea288aaac8387c5d1', 'request_id': 'req-aa844aba-d6d1-412e-9c37-6b127465d088'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:49.264435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9614129d1e864c3ea288aaac8387c5d1', 'request_id': 'req-aa844aba-d6d1-412e-9c37-6b127465d088'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:49.272923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9614129d1e864c3ea288aaac8387c5d1', 'request_id': 'req-aa844aba-d6d1-412e-9c37-6b127465d088'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:49.371049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] The token cache is being invalidate because project 9614129d1e864c3ea288aaac8387c5d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:16:49.371495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:49.372491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-aa844aba-d6d1-412e-9c37-6b127465d088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:49.374648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1355/2729] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:16:49 2022] DELETE /identity/v3/projects/9614129d1e864c3ea288aaac8387c5d1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:16:49.381105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322000-f4a6-459c-ae5c-06e6920a9bf0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:49.387339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322000-f4a6-459c-ae5c-06e6920a9bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:49.387339 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322000-f4a6-459c-ae5c-06e6920a9bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:49.455084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:49.471118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d0322000-f4a6-459c-ae5c-06e6920a9bf0 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:49.472931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d0322000-f4a6-459c-ae5c-06e6920a9bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:49.555299 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1375/2730] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:16:49.557155 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce97e2-dfc9-4ca1-a94e-075d8fb2dac8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:49.557444 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce97e2-dfc9-4ca1-a94e-075d8fb2dac8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:49.557731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce97e2-dfc9-4ca1-a94e-075d8fb2dac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:49.578862 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9dce97e2-dfc9-4ca1-a94e-075d8fb2dac8 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:49.580166 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9dce97e2-dfc9-4ca1-a94e-075d8fb2dac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:49.591863 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1356/2731] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:16:49.593177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0322000-f4a6-459c-ae5c-06e6920a9bf0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:49.609272 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0322000-f4a6-459c-ae5c-06e6920a9bf0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:49.617078 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:49.618501 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:49.618868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:49.619207 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:49.620040 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:16:49.620409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:49.622269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:49.639088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1376/2732] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 04:16:49.644923 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce97e2-dfc9-4ca1-a94e-075d8fb2dac8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:49.662421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce97e2-dfc9-4ca1-a94e-075d8fb2dac8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:49.668650 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfea1fe-edfe-42c4-ba4d-c1f2d7daf85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:49.670959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfea1fe-edfe-42c4-ba4d-c1f2d7daf85d admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:49.670959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfea1fe-edfe-42c4-ba4d-c1f2d7daf85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:49.670959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfea1fe-edfe-42c4-ba4d-c1f2d7daf85d admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:49.670959 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfea1fe-edfe-42c4-ba4d-c1f2d7daf85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:49.672889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfea1fe-edfe-42c4-ba4d-c1f2d7daf85d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:49.685347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:49.819798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1357/2733] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:16:49 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:16:49.821145 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:49.831874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e9753-2311-4d46-a83d-0cda340d0287 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:49.895462 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76533c-accc-444a-a47a-78cf9d5fdb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:49.896066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76533c-accc-444a-a47a-78cf9d5fdb4c admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:49.896346 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76533c-accc-444a-a47a-78cf9d5fdb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:49.896621 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76533c-accc-444a-a47a-78cf9d5fdb4c admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:49.897317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76533c-accc-444a-a47a-78cf9d5fdb4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:49.899788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76533c-accc-444a-a47a-78cf9d5fdb4c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:50.243573 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1377/2734] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:16:49 2022] POST /identity/v3/users => generated 284 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:16:50.247807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfea1fe-edfe-42c4-ba4d-c1f2d7daf85d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:50.258412 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfea1fe-edfe-42c4-ba4d-c1f2d7daf85d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:50.263531 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50c17c-168e-4f48-ad00-25eb46f51b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:50.264191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50c17c-168e-4f48-ad00-25eb46f51b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:50.264472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50c17c-168e-4f48-ad00-25eb46f51b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:50.264748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50c17c-168e-4f48-ad00-25eb46f51b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:50.265472 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50c17c-168e-4f48-ad00-25eb46f51b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:50.267049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50c17c-168e-4f48-ad00-25eb46f51b3a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:50.278640 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1358/2735] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:16:50.282943 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76533c-accc-444a-a47a-78cf9d5fdb4c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:50.293265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76533c-accc-444a-a47a-78cf9d5fdb4c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:50.298067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6d3b8-922a-4165-a282-f3a8528159c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:50.298724 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6d3b8-922a-4165-a282-f3a8528159c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:50.299007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6d3b8-922a-4165-a282-f3a8528159c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:50.299316 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6d3b8-922a-4165-a282-f3a8528159c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:50.300162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6d3b8-922a-4165-a282-f3a8528159c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:50.301762 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6d3b8-922a-4165-a282-f3a8528159c0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:50.311743 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1378/2736] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:16:50.319750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50c17c-168e-4f48-ad00-25eb46f51b3a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:50.332673 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50c17c-168e-4f48-ad00-25eb46f51b3a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:50.337587 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc354d-b3e7-45da-a0e4-4d36d2f67de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:50.338567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc354d-b3e7-45da-a0e4-4d36d2f67de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:50.338879 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc354d-b3e7-45da-a0e4-4d36d2f67de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:50.339173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc354d-b3e7-45da-a0e4-4d36d2f67de3 admin admin] PATH_INFO: `/v3/projects/afbb424a0b9c416e9abbb690a5bca7e4/users/c66c998f72b9415c95f13bf89dd6e0f8/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:50.358788 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc354d-b3e7-45da-a0e4-4d36d2f67de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbb424a0b9c416e9abbb690a5bca7e4, user_id=c66c998f72b9415c95f13bf89dd6e0f8, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:50.361646 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc354d-b3e7-45da-a0e4-4d36d2f67de3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:50.431992 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1359/2737] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:16:50 2022] PUT /identity/v3/projects/afbb424a0b9c416e9abbb690a5bca7e4/users/c66c998f72b9415c95f13bf89dd6e0f8/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:16:50.443399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fb7d4-d699-42a2-a6a3-651bdc52de18 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:50.444134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fb7d4-d699-42a2-a6a3-651bdc52de18 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:50.444234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fb7d4-d699-42a2-a6a3-651bdc52de18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:50.474367 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4b4fb7d4-d699-42a2-a6a3-651bdc52de18 None None] MFA Rules not processed for user `c66c998f72b9415c95f13bf89dd6e0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:50.475797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4b4fb7d4-d699-42a2-a6a3-651bdc52de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:50.562169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:50.568001 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1379/2738] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:16:50.573622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc354d-b3e7-45da-a0e4-4d36d2f67de3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:50.656195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc354d-b3e7-45da-a0e4-4d36d2f67de3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:50.661345 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7becdd-3e9f-422c-8752-a9134ac9d5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:50.662112 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7becdd-3e9f-422c-8752-a9134ac9d5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:50.662483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7becdd-3e9f-422c-8752-a9134ac9d5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:50.662805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7becdd-3e9f-422c-8752-a9134ac9d5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:50.663760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7becdd-3e9f-422c-8752-a9134ac9d5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:50.665845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7becdd-3e9f-422c-8752-a9134ac9d5c5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:50.761436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1360/2739] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:16:50 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:16:50.763007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fb7d4-d699-42a2-a6a3-651bdc52de18 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:50.775490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fb7d4-d699-42a2-a6a3-651bdc52de18 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:50.842041 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea4ab3-57a4-4482-ba97-c2e02644d15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:50.842869 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea4ab3-57a4-4482-ba97-c2e02644d15f admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:50.843158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea4ab3-57a4-4482-ba97-c2e02644d15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:50.843464 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea4ab3-57a4-4482-ba97-c2e02644d15f admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:50.844167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea4ab3-57a4-4482-ba97-c2e02644d15f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:50.846132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea4ab3-57a4-4482-ba97-c2e02644d15f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:50.948260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1380/2740] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:16:50 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:16:50.952604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7becdd-3e9f-422c-8752-a9134ac9d5c5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:50.968386 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7becdd-3e9f-422c-8752-a9134ac9d5c5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:50.976770 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471c3d9-7628-4bdc-8f8b-e08f47a264c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:50.977964 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471c3d9-7628-4bdc-8f8b-e08f47a264c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:50.978106 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471c3d9-7628-4bdc-8f8b-e08f47a264c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:50.978468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471c3d9-7628-4bdc-8f8b-e08f47a264c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:50.979269 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471c3d9-7628-4bdc-8f8b-e08f47a264c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:50.981446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471c3d9-7628-4bdc-8f8b-e08f47a264c0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:50.994917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1361/2741] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:16:50.999775 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea4ab3-57a4-4482-ba97-c2e02644d15f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:51.017022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea4ab3-57a4-4482-ba97-c2e02644d15f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:51.027117 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eae4cb-8d92-4b43-87b5-d125d4b74e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:51.028402 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eae4cb-8d92-4b43-87b5-d125d4b74e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:51.028819 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eae4cb-8d92-4b43-87b5-d125d4b74e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:51.029101 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eae4cb-8d92-4b43-87b5-d125d4b74e42 admin admin] PATH_INFO: `/v3/projects/2124c6f9b675443eb5326073c5ba60b7/users/45b6526fe4174927aa230e65f5e004f1/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:51.051586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eae4cb-8d92-4b43-87b5-d125d4b74e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2124c6f9b675443eb5326073c5ba60b7, user_id=45b6526fe4174927aa230e65f5e004f1, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:51.055095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eae4cb-8d92-4b43-87b5-d125d4b74e42 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:51.119254 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1381/2742] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:16:50 2022] PUT /identity/v3/projects/2124c6f9b675443eb5326073c5ba60b7/users/45b6526fe4174927aa230e65f5e004f1/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:16:51.126367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471c3d9-7628-4bdc-8f8b-e08f47a264c0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:51.138175 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471c3d9-7628-4bdc-8f8b-e08f47a264c0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:51.218943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dc0cd-8dac-49c3-8701-035021392cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:51.219724 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dc0cd-8dac-49c3-8701-035021392cee admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:51.220034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dc0cd-8dac-49c3-8701-035021392cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:51.220334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dc0cd-8dac-49c3-8701-035021392cee admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:51.221129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dc0cd-8dac-49c3-8701-035021392cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:51.222854 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dc0cd-8dac-49c3-8701-035021392cee admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:51.235315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1362/2743] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:16:51.239171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eae4cb-8d92-4b43-87b5-d125d4b74e42 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:51.252020 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eae4cb-8d92-4b43-87b5-d125d4b74e42 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:51.256340 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:51.257247 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:51.257664 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:51.257952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] PATH_INFO: `/v3/projects/2124c6f9b675443eb5326073c5ba60b7/users/45b6526fe4174927aa230e65f5e004f1/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:51.260584 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2124c6f9b675443eb5326073c5ba60b7, user_id=45b6526fe4174927aa230e65f5e004f1, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:51.263409 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:51.286131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1382/2744] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:16:51 2022] PUT /identity/v3/projects/2124c6f9b675443eb5326073c5ba60b7/users/45b6526fe4174927aa230e65f5e004f1/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:16:51.286991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630930-dd24-497a-958a-fbca4c425363 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:51.287499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630930-dd24-497a-958a-fbca4c425363 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:51.287610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630930-dd24-497a-958a-fbca4c425363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:51.317047 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-bb630930-dd24-497a-958a-fbca4c425363 None None] MFA Rules not processed for user `45b6526fe4174927aa230e65f5e004f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:51.318460 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bb630930-dd24-497a-958a-fbca4c425363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:51.335324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:51.407716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1363/2745] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:16:51.419797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:51.421912 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:51.488307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41263eca-5fde-44a6-be39-21077fcb3f74 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:51.492518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7964b6-8838-4068-a405-7321238437ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:51.493885 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7964b6-8838-4068-a405-7321238437ca service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:51.494167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7964b6-8838-4068-a405-7321238437ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:51.494444 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7964b6-8838-4068-a405-7321238437ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:51.505380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7964b6-8838-4068-a405-7321238437ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:51.506815 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7964b6-8838-4068-a405-7321238437ca service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:51.519761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1383/2746] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:16:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:16:51.632467 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb630930-dd24-497a-958a-fbca4c425363 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:51.634044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bb630930-dd24-497a-958a-fbca4c425363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:51.648342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb630930-dd24-497a-958a-fbca4c425363 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:51.652182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfeb04-07ac-4637-bfa6-d3187bd3875b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:51.652857 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeb04-07ac-4637-bfa6-d3187bd3875b service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:51.653190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeb04-07ac-4637-bfa6-d3187bd3875b service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:51.653660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeb04-07ac-4637-bfa6-d3187bd3875b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:51.735760 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfeb04-07ac-4637-bfa6-d3187bd3875b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:51.737287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfeb04-07ac-4637-bfa6-d3187bd3875b service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:51.748718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1364/2747] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:16:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:16:57.036849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7964b6-8838-4068-a405-7321238437ca service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:57.099081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:57.112928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7964b6-8838-4068-a405-7321238437ca service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:57.119439 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:57.120585 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:57.120863 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:57.121138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] PATH_INFO: `/v3/users/c66c998f72b9415c95f13bf89dd6e0f8` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:57.122831 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66c998f72b9415c95f13bf89dd6e0f8)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:57.124816 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:57.377429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66c998f72b9415c95f13bf89dd6e0f8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:57.378204 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66c998f72b9415c95f13bf89dd6e0f8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:57.378822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66c998f72b9415c95f13bf89dd6e0f8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:57.393208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66c998f72b9415c95f13bf89dd6e0f8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:57.400696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66c998f72b9415c95f13bf89dd6e0f8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:57.478565 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1384/2748] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:16:57 2022] DELETE /identity/v3/users/c66c998f72b9415c95f13bf89dd6e0f8 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:16:57.483036 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fff8e-d6dd-4e09-9130-dd0babbd2b09 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:57.483537 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fff8e-d6dd-4e09-9130-dd0babbd2b09 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:57.483715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fff8e-d6dd-4e09-9130-dd0babbd2b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:57.503366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a13fff8e-d6dd-4e09-9130-dd0babbd2b09 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:57.504620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a13fff8e-d6dd-4e09-9130-dd0babbd2b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:57.566114 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1365/2749] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:16:57.577186 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:57.580549 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:57.642425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dd10a-eedc-428c-87d5-761a0cd156bc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:57.646206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4f363-b4d1-4428-9802-96c2141ce119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:57.646856 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f363-b4d1-4428-9802-96c2141ce119 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:57.647140 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f363-b4d1-4428-9802-96c2141ce119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:57.647443 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4f363-b4d1-4428-9802-96c2141ce119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:57.656733 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4f363-b4d1-4428-9802-96c2141ce119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:57.658159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4f363-b4d1-4428-9802-96c2141ce119 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:57.668225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1385/2750] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:16:58.163606 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fff8e-d6dd-4e09-9130-dd0babbd2b09 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:58.165040 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a13fff8e-d6dd-4e09-9130-dd0babbd2b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:58.179398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fff8e-d6dd-4e09-9130-dd0babbd2b09 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:58.184150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:58.185007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:58.185289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:58.185562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] PATH_INFO: `/v3/projects/afbb424a0b9c416e9abbb690a5bca7e4` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:58.186738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afbb424a0b9c416e9abbb690a5bca7e4)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:58.189518 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:58.311792 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:58.315072 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afbb424a0b9c416e9abbb690a5bca7e4', 'request_id': 'req-89467426-28ab-4267-b2d5-fbbb2f0b8569'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.321321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afbb424a0b9c416e9abbb690a5bca7e4', 'request_id': 'req-89467426-28ab-4267-b2d5-fbbb2f0b8569'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.327772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afbb424a0b9c416e9abbb690a5bca7e4', 'request_id': 'req-89467426-28ab-4267-b2d5-fbbb2f0b8569'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.362177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6f584-e86d-4edb-b4bc-237ffd90c614 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:58.362644 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6f584-e86d-4edb-b4bc-237ffd90c614 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:58.362759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6f584-e86d-4edb-b4bc-237ffd90c614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:58.380624 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-eaa6f584-e86d-4edb-b4bc-237ffd90c614 None None] MFA Rules not processed for user `c41cca9ec5c94473b326e0940166ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:58.381837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eaa6f584-e86d-4edb-b4bc-237ffd90c614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:58.413048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] The token cache is being invalidate because project afbb424a0b9c416e9abbb690a5bca7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:16:58.413352 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.414073 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.415675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1366/2751] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:16:58 2022] DELETE /identity/v3/projects/afbb424a0b9c416e9abbb690a5bca7e4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:16:58.420148 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:58.421575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:58.441980 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1386/2752] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 1 04:16:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:16:58.454398 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6f584-e86d-4edb-b4bc-237ffd90c614 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:58.455789 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eaa6f584-e86d-4edb-b4bc-237ffd90c614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:58.478073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:58.489264 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467426-28ab-4267-b2d5-fbbb2f0b8569 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:58.494178 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:58.495043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:58.495358 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:58.495637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] PATH_INFO: `/v3/users/45b6526fe4174927aa230e65f5e004f1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:58.497034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b6526fe4174927aa230e65f5e004f1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:58.499095 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:58.529067 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6f584-e86d-4edb-b4bc-237ffd90c614 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:58.534829 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bbce5-76a8-49dc-87b0-72c2e0ec55ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123437cabbd4aa09ca3b1a0996a8bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:58.535667 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bbce5-76a8-49dc-87b0-72c2e0ec55ee service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:58.536048 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bbce5-76a8-49dc-87b0-72c2e0ec55ee service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:58.536435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bbce5-76a8-49dc-87b0-72c2e0ec55ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:58.545497 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bbce5-76a8-49dc-87b0-72c2e0ec55ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:58.547091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bbce5-76a8-49dc-87b0-72c2e0ec55ee service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:58.555948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1387/2753] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 1 04:16:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 04:16:58.760244 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b6526fe4174927aa230e65f5e004f1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.760991 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b6526fe4174927aa230e65f5e004f1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.853224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b6526fe4174927aa230e65f5e004f1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.861541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b6526fe4174927aa230e65f5e004f1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.862185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-06a6647f-8856-4867-8825-f25f136372a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b6526fe4174927aa230e65f5e004f1'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:58.879367 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1367/2754] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:16:58 2022] DELETE /identity/v3/users/45b6526fe4174927aa230e65f5e004f1 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:16:59.409873 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bbce5-76a8-49dc-87b0-72c2e0ec55ee service swift] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:59.425007 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bbce5-76a8-49dc-87b0-72c2e0ec55ee service swift] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:59.483270 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:59.484124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:59.484423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:59.484668 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] PATH_INFO: `/v3/projects/2124c6f9b675443eb5326073c5ba60b7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:59.485837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2124c6f9b675443eb5326073c5ba60b7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:59.487754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:59.654105 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2124c6f9b675443eb5326073c5ba60b7', 'request_id': 'req-712946d9-1035-488a-9010-5d2aa9ecbb2f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:59.659558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2124c6f9b675443eb5326073c5ba60b7', 'request_id': 'req-712946d9-1035-488a-9010-5d2aa9ecbb2f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:59.722434 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2124c6f9b675443eb5326073c5ba60b7', 'request_id': 'req-712946d9-1035-488a-9010-5d2aa9ecbb2f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:59.734860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] The token cache is being invalidate because project 2124c6f9b675443eb5326073c5ba60b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:16:59.735151 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:59.735803 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-712946d9-1035-488a-9010-5d2aa9ecbb2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:16:59.737396 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1388/2755] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:16:59 2022] DELETE /identity/v3/projects/2124c6f9b675443eb5326073c5ba60b7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:16:59.742268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16d2e-fd4f-4802-9160-071169b1b23c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:59.742732 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16d2e-fd4f-4802-9160-071169b1b23c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:59.742833 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16d2e-fd4f-4802-9160-071169b1b23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:59.760567 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e4c16d2e-fd4f-4802-9160-071169b1b23c None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:59.761716 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e4c16d2e-fd4f-4802-9160-071169b1b23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:59.810888 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:16:59.828247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1368/2756] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:16:59.836660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc625-32b4-4e6b-84c8-0e3582dc7cb4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:59.841366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc625-32b4-4e6b-84c8-0e3582dc7cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:59.841366 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc625-32b4-4e6b-84c8-0e3582dc7cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:59.856183 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9e6bc625-32b4-4e6b-84c8-0e3582dc7cb4 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:16:59.858099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9e6bc625-32b4-4e6b-84c8-0e3582dc7cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:16:59.866187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1389/2757] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:16:59.873376 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16d2e-fd4f-4802-9160-071169b1b23c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:59.880703 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16d2e-fd4f-4802-9160-071169b1b23c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:59.885406 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:59.886049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:59.886335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:59.886614 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:59.887417 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:16:59.888220 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:59.889805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:16:59.901678 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1369/2758] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:16:59.906813 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bc625-32b4-4e6b-84c8-0e3582dc7cb4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:16:59.916500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bc625-32b4-4e6b-84c8-0e3582dc7cb4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:16:59.921135 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c465bd5-e05e-4312-9695-57d5037e3370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:16:59.921782 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c465bd5-e05e-4312-9695-57d5037e3370 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:16:59.922063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c465bd5-e05e-4312-9695-57d5037e3370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:16:59.922336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c465bd5-e05e-4312-9695-57d5037e3370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:16:59.923023 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c465bd5-e05e-4312-9695-57d5037e3370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:16:59.924938 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c465bd5-e05e-4312-9695-57d5037e3370 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:00.106935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1390/2759] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:16:59 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:17:00.111266 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:00.121208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57c6d1-ec18-4a1a-ab3c-ee093515a372 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:00.178665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431023f-2aa3-410e-9773-5610b7ccc1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:00.179499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431023f-2aa3-410e-9773-5610b7ccc1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.179780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431023f-2aa3-410e-9773-5610b7ccc1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.180129 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431023f-2aa3-410e-9773-5610b7ccc1e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.180820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431023f-2aa3-410e-9773-5610b7ccc1e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:00.182715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431023f-2aa3-410e-9773-5610b7ccc1e1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:00.288207 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1370/2760] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:00 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:00.292359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c465bd5-e05e-4312-9695-57d5037e3370 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:00.302903 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c465bd5-e05e-4312-9695-57d5037e3370 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:00.307731 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150234bd-8d87-41f6-b2bd-852126e36173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:00.308399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150234bd-8d87-41f6-b2bd-852126e36173 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.308686 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150234bd-8d87-41f6-b2bd-852126e36173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.308968 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150234bd-8d87-41f6-b2bd-852126e36173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.310307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150234bd-8d87-41f6-b2bd-852126e36173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:00.311931 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150234bd-8d87-41f6-b2bd-852126e36173 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:00.317241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:00.323468 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1391/2761] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:00.328057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431023f-2aa3-410e-9773-5610b7ccc1e1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:00.338319 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431023f-2aa3-410e-9773-5610b7ccc1e1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:00.343157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9f2cc-9679-4c4b-ba97-441ec14e16aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:00.343860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9f2cc-9679-4c4b-ba97-441ec14e16aa admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.344143 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9f2cc-9679-4c4b-ba97-441ec14e16aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.344441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9f2cc-9679-4c4b-ba97-441ec14e16aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.345197 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9f2cc-9679-4c4b-ba97-441ec14e16aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:00.346793 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9f2cc-9679-4c4b-ba97-441ec14e16aa admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:00.358802 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1371/2762] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:00.363006 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150234bd-8d87-41f6-b2bd-852126e36173 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:00.374168 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150234bd-8d87-41f6-b2bd-852126e36173 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:00.379035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04a20a-8ef9-4bca-8bfb-4c336a596689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:00.380088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04a20a-8ef9-4bca-8bfb-4c336a596689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.380374 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04a20a-8ef9-4bca-8bfb-4c336a596689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.380652 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04a20a-8ef9-4bca-8bfb-4c336a596689 admin admin] PATH_INFO: `/v3/projects/81b0ed53da21454594f0c8e308abcb9f/users/7d6cee5bc8c34542b0f40503847d2a78/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.396416 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04a20a-8ef9-4bca-8bfb-4c336a596689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b0ed53da21454594f0c8e308abcb9f, user_id=7d6cee5bc8c34542b0f40503847d2a78, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:00.399068 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04a20a-8ef9-4bca-8bfb-4c336a596689 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:00.476945 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1392/2763] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:00 2022] PUT /identity/v3/projects/81b0ed53da21454594f0c8e308abcb9f/users/7d6cee5bc8c34542b0f40503847d2a78/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:17:00.488173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe303a58-e3af-4890-bc82-2dbf8ae68494 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.488473 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe303a58-e3af-4890-bc82-2dbf8ae68494 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.488763 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe303a58-e3af-4890-bc82-2dbf8ae68494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.517389 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fe303a58-e3af-4890-bc82-2dbf8ae68494 None None] MFA Rules not processed for user `7d6cee5bc8c34542b0f40503847d2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:00.518620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fe303a58-e3af-4890-bc82-2dbf8ae68494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:00.584383 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1372/2764] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:00.589080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04a20a-8ef9-4bca-8bfb-4c336a596689 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:00.652500 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04a20a-8ef9-4bca-8bfb-4c336a596689 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:00.657166 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b5fc0-f92f-47c3-9eb6-0cac4948d01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:00.657865 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b5fc0-f92f-47c3-9eb6-0cac4948d01e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.658149 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b5fc0-f92f-47c3-9eb6-0cac4948d01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.658424 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b5fc0-f92f-47c3-9eb6-0cac4948d01e admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.659109 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b5fc0-f92f-47c3-9eb6-0cac4948d01e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:00.661035 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b5fc0-f92f-47c3-9eb6-0cac4948d01e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:00.752053 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1393/2765] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:00 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:00.756168 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe303a58-e3af-4890-bc82-2dbf8ae68494 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:00.767157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe303a58-e3af-4890-bc82-2dbf8ae68494 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:00.812104 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:00.824966 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea2d0e-c69e-4429-b62a-e9cef5fb6be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:00.825632 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea2d0e-c69e-4429-b62a-e9cef5fb6be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.825916 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea2d0e-c69e-4429-b62a-e9cef5fb6be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.826192 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea2d0e-c69e-4429-b62a-e9cef5fb6be7 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.826883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea2d0e-c69e-4429-b62a-e9cef5fb6be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:00.829018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea2d0e-c69e-4429-b62a-e9cef5fb6be7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:00.927818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1373/2766] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:00 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:00.931890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b5fc0-f92f-47c3-9eb6-0cac4948d01e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:00.943090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b5fc0-f92f-47c3-9eb6-0cac4948d01e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:00.948080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911749-b413-4c76-81ff-4ff8b6dd2b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:00.948739 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911749-b413-4c76-81ff-4ff8b6dd2b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.949586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911749-b413-4c76-81ff-4ff8b6dd2b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.949586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911749-b413-4c76-81ff-4ff8b6dd2b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.950031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911749-b413-4c76-81ff-4ff8b6dd2b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:00.951660 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911749-b413-4c76-81ff-4ff8b6dd2b20 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:00.963192 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1394/2767] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:00.967343 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea2d0e-c69e-4429-b62a-e9cef5fb6be7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:00.977491 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea2d0e-c69e-4429-b62a-e9cef5fb6be7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:00.982694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8484ee1-9ba8-4cca-b3d3-7956cc0609fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:00.983600 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8484ee1-9ba8-4cca-b3d3-7956cc0609fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:00.983884 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8484ee1-9ba8-4cca-b3d3-7956cc0609fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:00.984158 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8484ee1-9ba8-4cca-b3d3-7956cc0609fe admin admin] PATH_INFO: `/v3/projects/3d7c864870dd4088bda986a0f11380f2/users/9989546b501542bd8848e6ec61f4d214/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:00.999870 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8484ee1-9ba8-4cca-b3d3-7956cc0609fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7c864870dd4088bda986a0f11380f2, user_id=9989546b501542bd8848e6ec61f4d214, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:01.002819 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8484ee1-9ba8-4cca-b3d3-7956cc0609fe admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:01.059410 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1374/2768] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:00 2022] PUT /identity/v3/projects/3d7c864870dd4088bda986a0f11380f2/users/9989546b501542bd8848e6ec61f4d214/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:01.063357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911749-b413-4c76-81ff-4ff8b6dd2b20 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:01.073726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911749-b413-4c76-81ff-4ff8b6dd2b20 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:01.128440 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5223c3-463b-4086-8765-9c48eb4017d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:01.129093 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5223c3-463b-4086-8765-9c48eb4017d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:01.129370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5223c3-463b-4086-8765-9c48eb4017d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:01.129642 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5223c3-463b-4086-8765-9c48eb4017d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:01.130360 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5223c3-463b-4086-8765-9c48eb4017d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:01.131973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5223c3-463b-4086-8765-9c48eb4017d0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:01.141430 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1395/2769] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:01.145727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8484ee1-9ba8-4cca-b3d3-7956cc0609fe admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:01.156569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8484ee1-9ba8-4cca-b3d3-7956cc0609fe admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:01.161179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:01.162052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:01.162338 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:01.162616 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] PATH_INFO: `/v3/projects/3d7c864870dd4088bda986a0f11380f2/users/9989546b501542bd8848e6ec61f4d214/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:01.164655 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7c864870dd4088bda986a0f11380f2, user_id=9989546b501542bd8848e6ec61f4d214, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:01.167287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:01.178111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1375/2770] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:01 2022] PUT /identity/v3/projects/3d7c864870dd4088bda986a0f11380f2/users/9989546b501542bd8848e6ec61f4d214/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:01.188112 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efb02e-ec38-4eca-8c93-b5bc661c5f97 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:01.188405 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efb02e-ec38-4eca-8c93-b5bc661c5f97 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:01.188687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efb02e-ec38-4eca-8c93-b5bc661c5f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:01.217060 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-81efb02e-ec38-4eca-8c93-b5bc661c5f97 None None] MFA Rules not processed for user `9989546b501542bd8848e6ec61f4d214`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:01.218278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-81efb02e-ec38-4eca-8c93-b5bc661c5f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:01.252524 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:01.288581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1396/2771] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:01.301017 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:01.302752 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:01.388901 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175a0da-cf25-4b36-9157-46a078c30145 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:01.394164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369103c6-2f34-4901-bf36-4b70e27a28f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:01.401076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369103c6-2f34-4901-bf36-4b70e27a28f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:01.401076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369103c6-2f34-4901-bf36-4b70e27a28f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:01.401076 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369103c6-2f34-4901-bf36-4b70e27a28f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:01.405505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369103c6-2f34-4901-bf36-4b70e27a28f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:01.406913 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369103c6-2f34-4901-bf36-4b70e27a28f1 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:01.416286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1376/2772] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:01.883765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efb02e-ec38-4eca-8c93-b5bc661c5f97 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:01.884835 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-81efb02e-ec38-4eca-8c93-b5bc661c5f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:01.952811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efb02e-ec38-4eca-8c93-b5bc661c5f97 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:01.957417 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:01.958315 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:01.958599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:01.958870 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] PATH_INFO: `/v3/users/90fc6c2d12f3420eb7d646d7fceb8965` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:01.960098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90fc6c2d12f3420eb7d646d7fceb8965)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:01.962711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:02.245031 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90fc6c2d12f3420eb7d646d7fceb8965'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.245609 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90fc6c2d12f3420eb7d646d7fceb8965'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.246227 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90fc6c2d12f3420eb7d646d7fceb8965'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.258911 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90fc6c2d12f3420eb7d646d7fceb8965'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.265874 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90fc6c2d12f3420eb7d646d7fceb8965'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.350113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1397/2773] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:17:01 2022] DELETE /identity/v3/users/90fc6c2d12f3420eb7d646d7fceb8965 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:02.486080 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369103c6-2f34-4901-bf36-4b70e27a28f1 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:02.495726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369103c6-2f34-4901-bf36-4b70e27a28f1 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:02.558011 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9989546b501542bd8848e6ec61f4d214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d7c864870dd4088bda986a0f11380f2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:02.558675 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:02.558961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:02.559292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:02.560025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:17:02.560347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:02.561995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:02.573967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1377/2774] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:17:02.578198 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:02.579761 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:02.593590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebac55c-5ad3-49ac-ba2d-29514e704fb1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:02.598079 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bf1bd-a9f3-459f-825d-3bd65f851af9 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9989546b501542bd8848e6ec61f4d214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d7c864870dd4088bda986a0f11380f2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:02.598729 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf1bd-a9f3-459f-825d-3bd65f851af9 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:02.599016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf1bd-a9f3-459f-825d-3bd65f851af9 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:02.599330 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf1bd-a9f3-459f-825d-3bd65f851af9 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:02.600032 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bf1bd-a9f3-459f-825d-3bd65f851af9 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:02.601969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bf1bd-a9f3-459f-825d-3bd65f851af9 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:02.620532 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:02.623039 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:02.645214 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:02.690127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101327e9-ab88-470c-add0-571288a2add4 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:02.695308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:02.696173 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:02.696454 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:02.696729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] PATH_INFO: `/v3/projects/a9f7e5a8b43740ab8cd82b939a8af457` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:02.697917 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f7e5a8b43740ab8cd82b939a8af457)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:02.699841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:02.705029 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1398/2775] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:02 2022] POST /identity/v3/projects => generated 355 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:02.706489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bf1bd-a9f3-459f-825d-3bd65f851af9 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:02.742167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:02.774350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bf1bd-a9f3-459f-825d-3bd65f851af9 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:02.779817 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9989546b501542bd8848e6ec61f4d214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d7c864870dd4088bda986a0f11380f2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:02.780484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:02.780774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:02.781055 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:02.781781 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:17:02.782095 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:02.783760 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:02.796459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1399/2776] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 04:17:02.810275 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f7e5a8b43740ab8cd82b939a8af457', 'request_id': 'req-0efcf5b1-b331-4d87-aaba-936685685bda'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.816169 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f7e5a8b43740ab8cd82b939a8af457', 'request_id': 'req-0efcf5b1-b331-4d87-aaba-936685685bda'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.823523 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f7e5a8b43740ab8cd82b939a8af457', 'request_id': 'req-0efcf5b1-b331-4d87-aaba-936685685bda'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.836159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:02.847307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3278c5-1349-4af4-929f-57f282e19fcb tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:02.905209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9989546b501542bd8848e6ec61f4d214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d7c864870dd4088bda986a0f11380f2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:02.906083 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:02.906370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:02.907280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] PATH_INFO: `/v3/projects/fce6bb144dd74074857ba692888c3c0f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:02.908521 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fce6bb144dd74074857ba692888c3c0f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:02.910512 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:02.910989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] The token cache is being invalidate because project a9f7e5a8b43740ab8cd82b939a8af457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:02.911315 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.911967 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0efcf5b1-b331-4d87-aaba-936685685bda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:02.913765 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1378/2777] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:02 2022] DELETE /identity/v3/projects/a9f7e5a8b43740ab8cd82b939a8af457 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:17:02.918240 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eb90e-35f3-4e31-a0d5-4165d5b8f493 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:02.918529 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eb90e-35f3-4e31-a0d5-4165d5b8f493 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:02.918921 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eb90e-35f3-4e31-a0d5-4165d5b8f493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:02.937769 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ac1eb90e-35f3-4e31-a0d5-4165d5b8f493 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:02.938993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ac1eb90e-35f3-4e31-a0d5-4165d5b8f493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:02.998151 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1379/2778] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:17:03.006495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a22ba1-90b9-416c-8579-e4f8dbe4b39f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.006789 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a22ba1-90b9-416c-8579-e4f8dbe4b39f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.007068 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a22ba1-90b9-416c-8579-e4f8dbe4b39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.026081 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-07a22ba1-90b9-416c-8579-e4f8dbe4b39f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:03.027234 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-07a22ba1-90b9-416c-8579-e4f8dbe4b39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:03.037613 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce6bb144dd74074857ba692888c3c0f', 'request_id': 'req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:03.044118 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce6bb144dd74074857ba692888c3c0f', 'request_id': 'req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:03.073221 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:03.089189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1380/2779] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:17:03.094222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a22ba1-90b9-416c-8579-e4f8dbe4b39f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:03.103179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a22ba1-90b9-416c-8579-e4f8dbe4b39f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:03.108013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:03.108726 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.109015 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.109292 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.112601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:17:03.112601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:03.112601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:03.123686 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1381/2780] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 04:17:03.127634 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce6bb144dd74074857ba692888c3c0f', 'request_id': 'req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:03.128035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:03.138666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861a7ef-8a61-4dc0-9f0c-0dbe614e3db8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:03.140249 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] The token cache is being invalidate because project fce6bb144dd74074857ba692888c3c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:03.140545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:03.141210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:03.144820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1400/2781] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:02 2022] DELETE /identity/v3/projects/fce6bb144dd74074857ba692888c3c0f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:17:03.144943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec50498-b079-4a33-b285-a6e1fd2d92e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:03.144943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec50498-b079-4a33-b285-a6e1fd2d92e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.145135 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec50498-b079-4a33-b285-a6e1fd2d92e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.145286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec50498-b079-4a33-b285-a6e1fd2d92e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.145989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec50498-b079-4a33-b285-a6e1fd2d92e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:03.148133 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec50498-b079-4a33-b285-a6e1fd2d92e1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:03.231422 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1382/2782] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:03 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:17:03.236094 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:03.237508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:03.251072 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab45fe-9ac7-4ca1-90b6-d4e70f564503 tempest-VolumeQuotaClassesTest-1109497456 tempest-VolumeQuotaClassesTest-1109497456-project-admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:03.299106 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:03.308343 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4edc2-4577-47f6-93ef-16dcef2164b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:03.309022 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4edc2-4577-47f6-93ef-16dcef2164b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.309849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4edc2-4577-47f6-93ef-16dcef2164b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.310125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4edc2-4577-47f6-93ef-16dcef2164b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.310820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4edc2-4577-47f6-93ef-16dcef2164b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:03.313528 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4edc2-4577-47f6-93ef-16dcef2164b1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:03.395470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1401/2783] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:03 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:03.399977 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec50498-b079-4a33-b285-a6e1fd2d92e1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:03.410538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec50498-b079-4a33-b285-a6e1fd2d92e1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:03.415512 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42743cab-05d6-4a11-89cc-e260ba0feafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:03.416184 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42743cab-05d6-4a11-89cc-e260ba0feafb admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.416569 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42743cab-05d6-4a11-89cc-e260ba0feafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.416851 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42743cab-05d6-4a11-89cc-e260ba0feafb admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.417682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42743cab-05d6-4a11-89cc-e260ba0feafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:03.419507 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42743cab-05d6-4a11-89cc-e260ba0feafb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:03.430064 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1383/2784] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:03.436544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4edc2-4577-47f6-93ef-16dcef2164b1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:03.445388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4edc2-4577-47f6-93ef-16dcef2164b1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:03.450321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e27bd-1488-4421-aa1d-c23b32f71545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:03.451947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e27bd-1488-4421-aa1d-c23b32f71545 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.452232 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e27bd-1488-4421-aa1d-c23b32f71545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.452507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e27bd-1488-4421-aa1d-c23b32f71545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.453241 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e27bd-1488-4421-aa1d-c23b32f71545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:03.454841 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e27bd-1488-4421-aa1d-c23b32f71545 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:03.465298 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1402/2785] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:03.470012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42743cab-05d6-4a11-89cc-e260ba0feafb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:03.481150 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42743cab-05d6-4a11-89cc-e260ba0feafb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:03.485620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:03.486495 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.488157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.488157 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] PATH_INFO: `/v3/projects/0ee795307db94dc6b6e4b066ac5814a0/users/818dbd27888e486dae4114b182519932/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.502474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee795307db94dc6b6e4b066ac5814a0, user_id=818dbd27888e486dae4114b182519932, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:03.505189 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:03.620020 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1384/2786] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:03 2022] PUT /identity/v3/projects/0ee795307db94dc6b6e4b066ac5814a0/users/818dbd27888e486dae4114b182519932/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:03.636693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb46f9f-f290-40c5-8d56-b3c1a17e4d76 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.636983 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb46f9f-f290-40c5-8d56-b3c1a17e4d76 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.637264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb46f9f-f290-40c5-8d56-b3c1a17e4d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.667487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8bb46f9f-f290-40c5-8d56-b3c1a17e4d76 None None] MFA Rules not processed for user `818dbd27888e486dae4114b182519932`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:03.668697 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8bb46f9f-f290-40c5-8d56-b3c1a17e4d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:03.737073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1403/2787] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:03.749273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:03.751043 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:03.810786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08ba0a-c372-4931-9d75-55ffeee615a4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:03.814808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754a70b-4fa8-4cd0-83c6-df49ab9eba33 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8327d9916424843b2a67cbdd856ad37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:03.815499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a70b-4fa8-4cd0-83c6-df49ab9eba33 service glance] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.815798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a70b-4fa8-4cd0-83c6-df49ab9eba33 service glance] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.816101 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a70b-4fa8-4cd0-83c6-df49ab9eba33 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.825620 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754a70b-4fa8-4cd0-83c6-df49ab9eba33 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:03.827049 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754a70b-4fa8-4cd0-83c6-df49ab9eba33 service glance] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:03.829944 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb46f9f-f290-40c5-8d56-b3c1a17e4d76 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:03.831425 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8bb46f9f-f290-40c5-8d56-b3c1a17e4d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:03.836617 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1385/2788] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:03.901518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb46f9f-f290-40c5-8d56-b3c1a17e4d76 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:03.906784 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:03.907900 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:03.908238 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:03.908519 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] PATH_INFO: `/v3/users/7d6cee5bc8c34542b0f40503847d2a78` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:03.909754 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6cee5bc8c34542b0f40503847d2a78)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:03.911757 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:04.101860 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6cee5bc8c34542b0f40503847d2a78'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:04.102999 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6cee5bc8c34542b0f40503847d2a78'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:04.104076 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6cee5bc8c34542b0f40503847d2a78'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:04.117899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:04.119205 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6cee5bc8c34542b0f40503847d2a78'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:04.127553 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6cee5bc8c34542b0f40503847d2a78'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:04.241932 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1404/2789] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:17:03 2022] DELETE /identity/v3/users/7d6cee5bc8c34542b0f40503847d2a78 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:04.246436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56d76-3326-46e7-8076-06105a1f400f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:04.246727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56d76-3326-46e7-8076-06105a1f400f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:04.247005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56d76-3326-46e7-8076-06105a1f400f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:04.264415 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6de56d76-3326-46e7-8076-06105a1f400f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:04.265623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6de56d76-3326-46e7-8076-06105a1f400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:04.326115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1386/2790] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:04.335778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:04.337385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:04.396806 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4225af-787c-43dc-89bf-8f133806e092 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:04.400381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e9aba-448c-4f8e-8bb8-0a3bf491b002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:04.401025 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e9aba-448c-4f8e-8bb8-0a3bf491b002 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:04.401306 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e9aba-448c-4f8e-8bb8-0a3bf491b002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:04.401581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e9aba-448c-4f8e-8bb8-0a3bf491b002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:04.411350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e9aba-448c-4f8e-8bb8-0a3bf491b002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:04.412776 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e9aba-448c-4f8e-8bb8-0a3bf491b002 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:04.422397 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1405/2791] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:04.844230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de56d76-3326-46e7-8076-06105a1f400f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:04.845831 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6de56d76-3326-46e7-8076-06105a1f400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:04.858492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de56d76-3326-46e7-8076-06105a1f400f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:04.863690 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:04.864557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:04.865335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:04.865335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] PATH_INFO: `/v3/projects/81b0ed53da21454594f0c8e308abcb9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:04.866359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b0ed53da21454594f0c8e308abcb9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:04.871395 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:04.886397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:04.974441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b0ed53da21454594f0c8e308abcb9f', 'request_id': 'req-ffa83312-c978-4073-85ea-8b2bf4ba60af'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:04.980643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b0ed53da21454594f0c8e308abcb9f', 'request_id': 'req-ffa83312-c978-4073-85ea-8b2bf4ba60af'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:04.987084 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b0ed53da21454594f0c8e308abcb9f', 'request_id': 'req-ffa83312-c978-4073-85ea-8b2bf4ba60af'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:05.088838 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] The token cache is being invalidate because project 81b0ed53da21454594f0c8e308abcb9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:05.089138 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:05.089937 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:05.092140 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1387/2792] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:04 2022] DELETE /identity/v3/projects/81b0ed53da21454594f0c8e308abcb9f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:05.096580 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e9aba-448c-4f8e-8bb8-0a3bf491b002 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:05.108319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e9aba-448c-4f8e-8bb8-0a3bf491b002 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:05.168074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:05.168950 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:05.169231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:05.169510 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] PATH_INFO: `/v3/users/9989546b501542bd8848e6ec61f4d214` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:05.171239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9989546b501542bd8848e6ec61f4d214)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:05.173147 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:05.416206 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9989546b501542bd8848e6ec61f4d214'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:05.416937 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9989546b501542bd8848e6ec61f4d214'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:05.417555 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9989546b501542bd8848e6ec61f4d214'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:05.430969 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9989546b501542bd8848e6ec61f4d214'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:05.438450 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cc3af808-7dc9-4f78-9366-0a8e60adc233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9989546b501542bd8848e6ec61f4d214'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:05.510818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1406/2793] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:17:05 2022] DELETE /identity/v3/users/9989546b501542bd8848e6ec61f4d214 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:17:05.897052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:05.898613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:05.913126 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa83312-c978-4073-85ea-8b2bf4ba60af admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:05.975468 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:05.976805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:05.977127 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:05.977579 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] PATH_INFO: `/v3/projects/3d7c864870dd4088bda986a0f11380f2` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:05.979008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7c864870dd4088bda986a0f11380f2)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:05.980954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:06.114458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7c864870dd4088bda986a0f11380f2', 'request_id': 'req-bc4fecc8-8e81-416e-ae8c-035db25dee25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:06.120883 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7c864870dd4088bda986a0f11380f2', 'request_id': 'req-bc4fecc8-8e81-416e-ae8c-035db25dee25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:06.129226 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7c864870dd4088bda986a0f11380f2', 'request_id': 'req-bc4fecc8-8e81-416e-ae8c-035db25dee25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:06.186746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] The token cache is being invalidate because project 3d7c864870dd4088bda986a0f11380f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:06.187048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:06.187755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bc4fecc8-8e81-416e-ae8c-035db25dee25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:06.189329 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1388/2794] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:05 2022] DELETE /identity/v3/projects/3d7c864870dd4088bda986a0f11380f2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:06.194666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4229a51-96ba-4c36-a54b-c48e2bf7de6d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.195161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4229a51-96ba-4c36-a54b-c48e2bf7de6d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.196534 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4229a51-96ba-4c36-a54b-c48e2bf7de6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.202155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:06.216070 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f4229a51-96ba-4c36-a54b-c48e2bf7de6d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:06.217763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f4229a51-96ba-4c36-a54b-c48e2bf7de6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:06.280978 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1407/2795] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:06.288718 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f725c-0247-4f7f-b525-abe816840c1e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.289008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f725c-0247-4f7f-b525-abe816840c1e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.289287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f725c-0247-4f7f-b525-abe816840c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.309420 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1b4f725c-0247-4f7f-b525-abe816840c1e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:06.311182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1b4f725c-0247-4f7f-b525-abe816840c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:06.319820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1389/2796] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:06.324059 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4229a51-96ba-4c36-a54b-c48e2bf7de6d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:06.334821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4229a51-96ba-4c36-a54b-c48e2bf7de6d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:06.340868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:06.341607 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.341890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.342167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.342910 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:17:06.343415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:06.344962 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:06.357623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1408/2797] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:17:06.361613 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f725c-0247-4f7f-b525-abe816840c1e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:06.371261 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f725c-0247-4f7f-b525-abe816840c1e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:06.377304 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f35d56-a921-4da0-924e-2f6b6bd5ec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:06.378053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35d56-a921-4da0-924e-2f6b6bd5ec24 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.378432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35d56-a921-4da0-924e-2f6b6bd5ec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.378805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35d56-a921-4da0-924e-2f6b6bd5ec24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.379666 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f35d56-a921-4da0-924e-2f6b6bd5ec24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:06.381670 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f35d56-a921-4da0-924e-2f6b6bd5ec24 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:06.512359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1390/2798] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:06 2022] POST /identity/v3/projects => generated 389 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:06.517341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:06.527834 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0fc0f-01e8-487b-b893-c1f0a04a5717 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:06.590038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e046b11-5e47-470c-9eb2-cb104f3f9939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:06.590722 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e046b11-5e47-470c-9eb2-cb104f3f9939 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.591009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e046b11-5e47-470c-9eb2-cb104f3f9939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.591320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e046b11-5e47-470c-9eb2-cb104f3f9939 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.592030 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e046b11-5e47-470c-9eb2-cb104f3f9939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:06.593960 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e046b11-5e47-470c-9eb2-cb104f3f9939 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:06.700308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1409/2799] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:06 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:06.706361 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f35d56-a921-4da0-924e-2f6b6bd5ec24 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:06.717115 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f35d56-a921-4da0-924e-2f6b6bd5ec24 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:06.721711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891266b3-996f-4e07-8001-359801c2a17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:06.722366 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891266b3-996f-4e07-8001-359801c2a17f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.722644 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891266b3-996f-4e07-8001-359801c2a17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.722920 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891266b3-996f-4e07-8001-359801c2a17f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.723672 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891266b3-996f-4e07-8001-359801c2a17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:06.725247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891266b3-996f-4e07-8001-359801c2a17f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:06.726608 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:06.736647 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1391/2800] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:06.740858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e046b11-5e47-470c-9eb2-cb104f3f9939 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:06.752297 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e046b11-5e47-470c-9eb2-cb104f3f9939 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:06.757320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420a321-6a1c-45d8-8b47-5c0e1a1a544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:06.758028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420a321-6a1c-45d8-8b47-5c0e1a1a544c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.758312 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420a321-6a1c-45d8-8b47-5c0e1a1a544c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.758590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420a321-6a1c-45d8-8b47-5c0e1a1a544c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.759382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420a321-6a1c-45d8-8b47-5c0e1a1a544c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:06.761150 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420a321-6a1c-45d8-8b47-5c0e1a1a544c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:06.772381 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1410/2801] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:06.775823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891266b3-996f-4e07-8001-359801c2a17f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:06.785731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891266b3-996f-4e07-8001-359801c2a17f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:06.791443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c424d-c902-4e04-ad5c-d972ddc6eebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:06.792382 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c424d-c902-4e04-ad5c-d972ddc6eebf admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.792907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c424d-c902-4e04-ad5c-d972ddc6eebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.793334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c424d-c902-4e04-ad5c-d972ddc6eebf admin admin] PATH_INFO: `/v3/projects/fea6711d12524575b06f6849300ec5c7/users/ac72956889c24badaeceb0bfd27c9a61/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.809576 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c424d-c902-4e04-ad5c-d972ddc6eebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea6711d12524575b06f6849300ec5c7, user_id=ac72956889c24badaeceb0bfd27c9a61, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:06.812891 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c424d-c902-4e04-ad5c-d972ddc6eebf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:06.933950 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1392/2802] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:06 2022] PUT /identity/v3/projects/fea6711d12524575b06f6849300ec5c7/users/ac72956889c24badaeceb0bfd27c9a61/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:06.946137 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa595ad6-df95-4c51-acd8-405664808d3d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:06.946453 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa595ad6-df95-4c51-acd8-405664808d3d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:06.946735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa595ad6-df95-4c51-acd8-405664808d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:06.974759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-aa595ad6-df95-4c51-acd8-405664808d3d None None] MFA Rules not processed for user `ac72956889c24badaeceb0bfd27c9a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:06.976057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-aa595ad6-df95-4c51-acd8-405664808d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:07.038610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:07.043904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1411/2803] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:07.048953 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c424d-c902-4e04-ad5c-d972ddc6eebf admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:07.119795 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c424d-c902-4e04-ad5c-d972ddc6eebf admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:07.124398 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7c8f-709a-467e-b203-ba96ee6dd1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:07.125048 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7c8f-709a-467e-b203-ba96ee6dd1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:07.125327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7c8f-709a-467e-b203-ba96ee6dd1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:07.125601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7c8f-709a-467e-b203-ba96ee6dd1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:07.126351 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb7c8f-709a-467e-b203-ba96ee6dd1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:07.128262 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb7c8f-709a-467e-b203-ba96ee6dd1a1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:07.265012 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1393/2804] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:07 2022] POST /identity/v3/projects => generated 393 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:17:07.266113 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa595ad6-df95-4c51-acd8-405664808d3d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:07.276849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa595ad6-df95-4c51-acd8-405664808d3d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:07.335483 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94595205-953a-4897-960a-3ba5a77ee292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:07.336061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94595205-953a-4897-960a-3ba5a77ee292 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:07.336344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94595205-953a-4897-960a-3ba5a77ee292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:07.336623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94595205-953a-4897-960a-3ba5a77ee292 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:07.337321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94595205-953a-4897-960a-3ba5a77ee292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:07.339263 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94595205-953a-4897-960a-3ba5a77ee292 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:07.440669 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1412/2805] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:07 2022] POST /identity/v3/users => generated 303 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:17:07.442813 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7c8f-709a-467e-b203-ba96ee6dd1a1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:07.454333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7c8f-709a-467e-b203-ba96ee6dd1a1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:07.458989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d5ee5-2c11-4315-a29c-8c9e35728ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:07.459938 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5ee5-2c11-4315-a29c-8c9e35728ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:07.460179 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5ee5-2c11-4315-a29c-8c9e35728ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:07.460289 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5ee5-2c11-4315-a29c-8c9e35728ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:07.460954 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d5ee5-2c11-4315-a29c-8c9e35728ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:07.462572 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d5ee5-2c11-4315-a29c-8c9e35728ff0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:07.474419 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1394/2806] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:07.478878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94595205-953a-4897-960a-3ba5a77ee292 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:07.489732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94595205-953a-4897-960a-3ba5a77ee292 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:07.494599 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de66db3-949a-4d7f-ac2e-ec5d0a2a7291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:07.495516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de66db3-949a-4d7f-ac2e-ec5d0a2a7291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:07.495796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de66db3-949a-4d7f-ac2e-ec5d0a2a7291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:07.496071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de66db3-949a-4d7f-ac2e-ec5d0a2a7291 admin admin] PATH_INFO: `/v3/projects/8b93b7a3b9e747c2a6524754bb6e633c/users/a9d9b996adf84e6597b844c7996cafff/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:07.512226 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de66db3-949a-4d7f-ac2e-ec5d0a2a7291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b93b7a3b9e747c2a6524754bb6e633c, user_id=a9d9b996adf84e6597b844c7996cafff, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:07.515768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de66db3-949a-4d7f-ac2e-ec5d0a2a7291 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:07.590878 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1413/2807] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:07 2022] PUT /identity/v3/projects/8b93b7a3b9e747c2a6524754bb6e633c/users/a9d9b996adf84e6597b844c7996cafff/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:07.595489 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d5ee5-2c11-4315-a29c-8c9e35728ff0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:07.606848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d5ee5-2c11-4315-a29c-8c9e35728ff0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:07.666914 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada287fc-f277-42c2-9bcc-3e09e3812ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:07.667619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada287fc-f277-42c2-9bcc-3e09e3812ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:07.667907 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada287fc-f277-42c2-9bcc-3e09e3812ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:07.668186 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada287fc-f277-42c2-9bcc-3e09e3812ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:07.668915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada287fc-f277-42c2-9bcc-3e09e3812ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:07.670542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada287fc-f277-42c2-9bcc-3e09e3812ca8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:07.680811 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1395/2808] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:07.684768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de66db3-949a-4d7f-ac2e-ec5d0a2a7291 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:07.697111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de66db3-949a-4d7f-ac2e-ec5d0a2a7291 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:07.703016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:07.703966 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:07.704244 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:07.704518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] PATH_INFO: `/v3/projects/8b93b7a3b9e747c2a6524754bb6e633c/users/a9d9b996adf84e6597b844c7996cafff/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:07.706558 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b93b7a3b9e747c2a6524754bb6e633c, user_id=a9d9b996adf84e6597b844c7996cafff, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:07.709333 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:07.721279 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1414/2809] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:07 2022] PUT /identity/v3/projects/8b93b7a3b9e747c2a6524754bb6e633c/users/a9d9b996adf84e6597b844c7996cafff/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:07.731416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f70cb-e894-43ad-8557-a02da662e0e6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:07.731709 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f70cb-e894-43ad-8557-a02da662e0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:07.732231 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f70cb-e894-43ad-8557-a02da662e0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:07.759124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3f2f70cb-e894-43ad-8557-a02da662e0e6 None None] MFA Rules not processed for user `a9d9b996adf84e6597b844c7996cafff`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:07.760432 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3f2f70cb-e894-43ad-8557-a02da662e0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:07.792425 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:07.833324 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1396/2810] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:07.844265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:07.845957 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:07.908129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2455d-9626-45bc-a429-d9d69d5c6575 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:07.912338 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae688c-c639-4fcb-a96d-f1c03584649f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:07.913047 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae688c-c639-4fcb-a96d-f1c03584649f service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:07.913494 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae688c-c639-4fcb-a96d-f1c03584649f service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:07.913774 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae688c-c639-4fcb-a96d-f1c03584649f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:07.923538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae688c-c639-4fcb-a96d-f1c03584649f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:07.925167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae688c-c639-4fcb-a96d-f1c03584649f service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:07.935162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1415/2811] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:07.983211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f70cb-e894-43ad-8557-a02da662e0e6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:07.984708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3f2f70cb-e894-43ad-8557-a02da662e0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:07.997602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f70cb-e894-43ad-8557-a02da662e0e6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:08.001798 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab62317-56f9-4823-ade6-eecb2280a66d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:08.002443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab62317-56f9-4823-ade6-eecb2280a66d service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:08.002728 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab62317-56f9-4823-ade6-eecb2280a66d service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:08.003007 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab62317-56f9-4823-ade6-eecb2280a66d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:08.061622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab62317-56f9-4823-ade6-eecb2280a66d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:08.063054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab62317-56f9-4823-ade6-eecb2280a66d service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:08.072605 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1397/2812] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:12.777223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae688c-c639-4fcb-a96d-f1c03584649f service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:12.830122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:12.840895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae688c-c639-4fcb-a96d-f1c03584649f service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:12.845370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:12.846228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:12.846507 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:12.846778 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] PATH_INFO: `/v3/users/ac72956889c24badaeceb0bfd27c9a61` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:12.847987 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac72956889c24badaeceb0bfd27c9a61)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:12.849914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:13.055820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac72956889c24badaeceb0bfd27c9a61'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:13.056470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac72956889c24badaeceb0bfd27c9a61'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:13.057074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac72956889c24badaeceb0bfd27c9a61'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:13.070043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac72956889c24badaeceb0bfd27c9a61'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:13.078086 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac72956889c24badaeceb0bfd27c9a61'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:13.174024 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1416/2813] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:17:12 2022] DELETE /identity/v3/users/ac72956889c24badaeceb0bfd27c9a61 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:13.178708 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b89061-0df7-4265-9c1f-2c2e28cbe44f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:13.183740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b89061-0df7-4265-9c1f-2c2e28cbe44f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:13.183740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b89061-0df7-4265-9c1f-2c2e28cbe44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:13.197497 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f1b89061-0df7-4265-9c1f-2c2e28cbe44f None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:13.198711 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f1b89061-0df7-4265-9c1f-2c2e28cbe44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:13.259746 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1398/2814] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:13.269419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:13.270821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:13.330036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3a483-57cf-4e50-aa4f-4c3b4ceaeb04 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:13.333985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3b226-4f4d-4b73-9ce2-961526060b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:13.334689 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3b226-4f4d-4b73-9ce2-961526060b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:13.334970 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3b226-4f4d-4b73-9ce2-961526060b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:13.335268 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3b226-4f4d-4b73-9ce2-961526060b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:13.345787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3b226-4f4d-4b73-9ce2-961526060b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:13.347260 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3b226-4f4d-4b73-9ce2-961526060b2f service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:13.356877 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1417/2815] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:13.814022 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b89061-0df7-4265-9c1f-2c2e28cbe44f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:13.815573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f1b89061-0df7-4265-9c1f-2c2e28cbe44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:13.831247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b89061-0df7-4265-9c1f-2c2e28cbe44f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:13.833886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:13.834748 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:13.835033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:13.835342 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] PATH_INFO: `/v3/projects/fea6711d12524575b06f6849300ec5c7` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:13.836540 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea6711d12524575b06f6849300ec5c7)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:13.838445 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:13.951578 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea6711d12524575b06f6849300ec5c7', 'request_id': 'req-853a05c0-b919-48de-b525-1fbea029fe96'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:13.957687 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea6711d12524575b06f6849300ec5c7', 'request_id': 'req-853a05c0-b919-48de-b525-1fbea029fe96'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:13.959575 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:13.964453 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea6711d12524575b06f6849300ec5c7', 'request_id': 'req-853a05c0-b919-48de-b525-1fbea029fe96'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.035694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] The token cache is being invalidate because project fea6711d12524575b06f6849300ec5c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:14.036019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.036665 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.038193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1399/2816] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:13 2022] DELETE /identity/v3/projects/fea6711d12524575b06f6849300ec5c7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:14.042337 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3b226-4f4d-4b73-9ce2-961526060b2f service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:14.052478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3b226-4f4d-4b73-9ce2-961526060b2f service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:14.111341 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:14.112208 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:14.112485 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:14.112758 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] PATH_INFO: `/v3/users/a9d9b996adf84e6597b844c7996cafff` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:14.113984 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d9b996adf84e6597b844c7996cafff)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:14.115973 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:14.132295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:14.277908 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d9b996adf84e6597b844c7996cafff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.278559 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d9b996adf84e6597b844c7996cafff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.279159 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d9b996adf84e6597b844c7996cafff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.291489 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d9b996adf84e6597b844c7996cafff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.299128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1d50f1f0-c7ce-4b15-8c1c-3fd83413d0b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d9b996adf84e6597b844c7996cafff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.350240 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1418/2817] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:17:14 2022] DELETE /identity/v3/users/a9d9b996adf84e6597b844c7996cafff => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:14.757547 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:14.759045 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:14.772466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a05c0-b919-48de-b525-1fbea029fe96 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:14.831141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:14.832035 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:14.832320 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:14.832597 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] PATH_INFO: `/v3/projects/8b93b7a3b9e747c2a6524754bb6e633c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:14.833775 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b93b7a3b9e747c2a6524754bb6e633c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:14.835845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:14.963136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b93b7a3b9e747c2a6524754bb6e633c', 'request_id': 'req-9c447720-790c-4bd0-ba48-f9950484caa6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.969230 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b93b7a3b9e747c2a6524754bb6e633c', 'request_id': 'req-9c447720-790c-4bd0-ba48-f9950484caa6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:14.975470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b93b7a3b9e747c2a6524754bb6e633c', 'request_id': 'req-9c447720-790c-4bd0-ba48-f9950484caa6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:15.067796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] The token cache is being invalidate because project 8b93b7a3b9e747c2a6524754bb6e633c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:15.067796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:15.067796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9c447720-790c-4bd0-ba48-f9950484caa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:15.067796 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1400/2818] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:14 2022] DELETE /identity/v3/projects/8b93b7a3b9e747c2a6524754bb6e633c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:15.071857 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d6d5b-8bf5-41ae-a3bc-0e61ced9e03d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.072344 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d6d5b-8bf5-41ae-a3bc-0e61ced9e03d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.072477 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d6d5b-8bf5-41ae-a3bc-0e61ced9e03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.091475 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e93d6d5b-8bf5-41ae-a3bc-0e61ced9e03d None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:15.092684 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e93d6d5b-8bf5-41ae-a3bc-0e61ced9e03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:15.152209 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1419/2819] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:15.215550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390030a-5d69-4462-b207-e6f36d18d364 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.215750 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390030a-5d69-4462-b207-e6f36d18d364 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.216004 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390030a-5d69-4462-b207-e6f36d18d364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.234490 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2390030a-5d69-4462-b207-e6f36d18d364 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:15.235757 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2390030a-5d69-4462-b207-e6f36d18d364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:15.243889 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1401/2820] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:15.248177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d6d5b-8bf5-41ae-a3bc-0e61ced9e03d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:15.258574 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d6d5b-8bf5-41ae-a3bc-0e61ced9e03d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:15.263210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:15.263899 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.264185 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.264465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.265842 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:17:15.266171 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:15.267820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:15.279618 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1420/2821] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:17:15.283977 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390030a-5d69-4462-b207-e6f36d18d364 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:15.293364 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390030a-5d69-4462-b207-e6f36d18d364 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:15.298260 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f211-0ef0-4cd4-af54-d26f270aae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:15.298905 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f211-0ef0-4cd4-af54-d26f270aae98 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.299190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f211-0ef0-4cd4-af54-d26f270aae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.299501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f211-0ef0-4cd4-af54-d26f270aae98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.300307 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f211-0ef0-4cd4-af54-d26f270aae98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:15.302190 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f211-0ef0-4cd4-af54-d26f270aae98 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:15.455738 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1402/2822] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:15 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:15.459948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:15.470177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2e8ca-b9c3-405b-a6bd-626c94e69fa0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:15.526640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5387bc4-a492-486e-938a-b97b22c941fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:15.527350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5387bc4-a492-486e-938a-b97b22c941fc admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.527654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5387bc4-a492-486e-938a-b97b22c941fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.527953 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5387bc4-a492-486e-938a-b97b22c941fc admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.528636 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5387bc4-a492-486e-938a-b97b22c941fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:15.530538 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5387bc4-a492-486e-938a-b97b22c941fc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:15.642084 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1421/2823] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:15 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:17:15.646323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f211-0ef0-4cd4-af54-d26f270aae98 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:15.656721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f211-0ef0-4cd4-af54-d26f270aae98 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:15.661285 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45b260-7bca-48bf-a690-c7a2e2df4d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:15.661931 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b260-7bca-48bf-a690-c7a2e2df4d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.662213 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b260-7bca-48bf-a690-c7a2e2df4d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.662484 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b260-7bca-48bf-a690-c7a2e2df4d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.663436 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45b260-7bca-48bf-a690-c7a2e2df4d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:15.667641 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45b260-7bca-48bf-a690-c7a2e2df4d6c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:15.675951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1403/2824] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:15.680004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5387bc4-a492-486e-938a-b97b22c941fc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:15.690666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5387bc4-a492-486e-938a-b97b22c941fc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:15.695641 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bde61-d367-4bb9-ad94-57766b35f21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:15.698581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bde61-d367-4bb9-ad94-57766b35f21a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.698581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bde61-d367-4bb9-ad94-57766b35f21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.698581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bde61-d367-4bb9-ad94-57766b35f21a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.698581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bde61-d367-4bb9-ad94-57766b35f21a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:15.699251 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bde61-d367-4bb9-ad94-57766b35f21a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:15.706009 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:15.710787 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1422/2825] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:17:15.715025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45b260-7bca-48bf-a690-c7a2e2df4d6c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:15.725296 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45b260-7bca-48bf-a690-c7a2e2df4d6c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:15.730203 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce41e41-d70f-4635-be15-a0b46a52237c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:15.731124 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce41e41-d70f-4635-be15-a0b46a52237c admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.731474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce41e41-d70f-4635-be15-a0b46a52237c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.731736 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce41e41-d70f-4635-be15-a0b46a52237c admin admin] PATH_INFO: `/v3/projects/fb17cb4253ac4cb68aa3357ca757c8ea/users/9f32cdde887c43a696c8635c9cf07dbd/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.747164 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce41e41-d70f-4635-be15-a0b46a52237c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb17cb4253ac4cb68aa3357ca757c8ea, user_id=9f32cdde887c43a696c8635c9cf07dbd, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:15.750501 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce41e41-d70f-4635-be15-a0b46a52237c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:15.824232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1404/2826] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:15 2022] PUT /identity/v3/projects/fb17cb4253ac4cb68aa3357ca757c8ea/users/9f32cdde887c43a696c8635c9cf07dbd/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:15.834650 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612c64-61d3-4438-9275-aac6ff560440 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:15.834961 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612c64-61d3-4438-9275-aac6ff560440 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:15.835265 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612c64-61d3-4438-9275-aac6ff560440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:15.861687 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ab612c64-61d3-4438-9275-aac6ff560440 None None] MFA Rules not processed for user `9f32cdde887c43a696c8635c9cf07dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:15.862947 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ab612c64-61d3-4438-9275-aac6ff560440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:15.929016 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1423/2827] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:15.933635 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce41e41-d70f-4635-be15-a0b46a52237c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:15.952016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:15.995948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce41e41-d70f-4635-be15-a0b46a52237c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:16.000729 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eaa93e-0aba-48c0-a8a2-b0dc0f5ebac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:16.006993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eaa93e-0aba-48c0-a8a2-b0dc0f5ebac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:16.006993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eaa93e-0aba-48c0-a8a2-b0dc0f5ebac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:16.006993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eaa93e-0aba-48c0-a8a2-b0dc0f5ebac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:16.006993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eaa93e-0aba-48c0-a8a2-b0dc0f5ebac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:16.006993 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eaa93e-0aba-48c0-a8a2-b0dc0f5ebac6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:16.109648 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1405/2828] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:15 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:16.115108 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab612c64-61d3-4438-9275-aac6ff560440 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:16.122435 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab612c64-61d3-4438-9275-aac6ff560440 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:16.179853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1c98f-79b6-43f8-93b7-df3239ea00d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:16.180526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1c98f-79b6-43f8-93b7-df3239ea00d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:16.180818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1c98f-79b6-43f8-93b7-df3239ea00d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:16.181091 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1c98f-79b6-43f8-93b7-df3239ea00d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:16.181794 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1c98f-79b6-43f8-93b7-df3239ea00d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:16.183747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1c98f-79b6-43f8-93b7-df3239ea00d9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:16.282267 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1424/2829] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:16 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:16.286370 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eaa93e-0aba-48c0-a8a2-b0dc0f5ebac6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:16.297623 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eaa93e-0aba-48c0-a8a2-b0dc0f5ebac6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:16.302435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3756a6a-8992-474b-9b2d-52bd4ab4798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:16.303199 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756a6a-8992-474b-9b2d-52bd4ab4798a admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:16.303538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756a6a-8992-474b-9b2d-52bd4ab4798a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:16.303818 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756a6a-8992-474b-9b2d-52bd4ab4798a admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:16.304591 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3756a6a-8992-474b-9b2d-52bd4ab4798a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:16.306177 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3756a6a-8992-474b-9b2d-52bd4ab4798a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:16.316958 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1406/2830] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:16.321451 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1c98f-79b6-43f8-93b7-df3239ea00d9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:16.332103 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1c98f-79b6-43f8-93b7-df3239ea00d9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:16.336648 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a0e95-78eb-45e9-839c-cc08845f72ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:16.337518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0e95-78eb-45e9-839c-cc08845f72ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:16.337801 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0e95-78eb-45e9-839c-cc08845f72ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:16.338077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0e95-78eb-45e9-839c-cc08845f72ed admin admin] PATH_INFO: `/v3/projects/0c3bf8d94f05450f810b8e7e9d313834/users/556171033520418a83b5f66a13f5f777/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:16.353277 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a0e95-78eb-45e9-839c-cc08845f72ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c3bf8d94f05450f810b8e7e9d313834, user_id=556171033520418a83b5f66a13f5f777, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:16.358572 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a0e95-78eb-45e9-839c-cc08845f72ed admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:16.442336 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1425/2831] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:16 2022] PUT /identity/v3/projects/0c3bf8d94f05450f810b8e7e9d313834/users/556171033520418a83b5f66a13f5f777/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:16.446622 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3756a6a-8992-474b-9b2d-52bd4ab4798a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:16.457117 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3756a6a-8992-474b-9b2d-52bd4ab4798a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:16.513220 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83b212-3c58-4f7b-89b5-4435816c7b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:16.513892 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83b212-3c58-4f7b-89b5-4435816c7b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:16.514176 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83b212-3c58-4f7b-89b5-4435816c7b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:16.514455 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83b212-3c58-4f7b-89b5-4435816c7b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:16.515816 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83b212-3c58-4f7b-89b5-4435816c7b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:16.517414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83b212-3c58-4f7b-89b5-4435816c7b01 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:16.527860 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1407/2832] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:16.532640 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a0e95-78eb-45e9-839c-cc08845f72ed admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:16.542884 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a0e95-78eb-45e9-839c-cc08845f72ed admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:16.547822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:16.548705 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:16.548990 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:16.549266 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] PATH_INFO: `/v3/projects/0c3bf8d94f05450f810b8e7e9d313834/users/556171033520418a83b5f66a13f5f777/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:16.551280 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c3bf8d94f05450f810b8e7e9d313834, user_id=556171033520418a83b5f66a13f5f777, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:16.553928 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:16.564606 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1426/2833] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:16 2022] PUT /identity/v3/projects/0c3bf8d94f05450f810b8e7e9d313834/users/556171033520418a83b5f66a13f5f777/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:16.575493 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ae65c-461f-4459-b9e2-aa577ac236b2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:16.575799 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ae65c-461f-4459-b9e2-aa577ac236b2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:16.576079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ae65c-461f-4459-b9e2-aa577ac236b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:16.604447 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-287ae65c-461f-4459-b9e2-aa577ac236b2 None None] MFA Rules not processed for user `556171033520418a83b5f66a13f5f777`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:16.605722 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-287ae65c-461f-4459-b9e2-aa577ac236b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:16.673078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1408/2834] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:16.684328 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:16.686011 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:16.733353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:16.744933 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481070e7-05af-4684-9542-a31aa53797da admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:16.748682 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249cf2f-108f-43f3-9aed-929e4d655b96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:16.749324 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cf2f-108f-43f3-9aed-929e4d655b96 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:16.750119 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cf2f-108f-43f3-9aed-929e4d655b96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:16.750399 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cf2f-108f-43f3-9aed-929e4d655b96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:16.759487 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249cf2f-108f-43f3-9aed-929e4d655b96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:16.760916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249cf2f-108f-43f3-9aed-929e4d655b96 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:16.771837 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1427/2835] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:17.061441 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ae65c-461f-4459-b9e2-aa577ac236b2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:17.071914 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ae65c-461f-4459-b9e2-aa577ac236b2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:17.075629 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cb2d1-ef81-46e1-a6f7-f3be5284437c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:17.076268 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cb2d1-ef81-46e1-a6f7-f3be5284437c service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:17.076550 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cb2d1-ef81-46e1-a6f7-f3be5284437c service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:17.076962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cb2d1-ef81-46e1-a6f7-f3be5284437c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:17.086435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:17.134603 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cb2d1-ef81-46e1-a6f7-f3be5284437c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:17.136060 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cb2d1-ef81-46e1-a6f7-f3be5284437c service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:17.145217 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1409/2836] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:17 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:23.483699 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249cf2f-108f-43f3-9aed-929e4d655b96 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:23.546887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249cf2f-108f-43f3-9aed-929e4d655b96 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:23.551562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:23.552465 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:23.552747 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:23.553043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] PATH_INFO: `/v3/users/9f32cdde887c43a696c8635c9cf07dbd` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:23.554239 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f32cdde887c43a696c8635c9cf07dbd)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:23.556218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:23.707812 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f32cdde887c43a696c8635c9cf07dbd'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:23.708478 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f32cdde887c43a696c8635c9cf07dbd'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:23.709210 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f32cdde887c43a696c8635c9cf07dbd'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:23.721390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f32cdde887c43a696c8635c9cf07dbd'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:23.728202 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f32cdde887c43a696c8635c9cf07dbd'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:23.778414 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1428/2837] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:17:23 2022] DELETE /identity/v3/users/9f32cdde887c43a696c8635c9cf07dbd => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:17:23.783134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de661de-d850-4220-8801-5e76683af48e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:23.783680 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de661de-d850-4220-8801-5e76683af48e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:23.783820 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de661de-d850-4220-8801-5e76683af48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:23.801581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7de661de-d850-4220-8801-5e76683af48e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:23.803390 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7de661de-d850-4220-8801-5e76683af48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:23.863945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1410/2838] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:23.874169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:23.875608 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:23.933196 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb654f90-b18a-445e-ac76-7d8f4ec880a7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:23.937081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352fca5-81c5-42f9-abd2-3145ec121e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:23.937726 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352fca5-81c5-42f9-abd2-3145ec121e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:23.938015 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352fca5-81c5-42f9-abd2-3145ec121e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:23.938290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352fca5-81c5-42f9-abd2-3145ec121e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:23.948228 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352fca5-81c5-42f9-abd2-3145ec121e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:23.949656 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352fca5-81c5-42f9-abd2-3145ec121e81 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:23.959148 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1429/2839] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:24.362755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de661de-d850-4220-8801-5e76683af48e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:24.364589 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7de661de-d850-4220-8801-5e76683af48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:24.377804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de661de-d850-4220-8801-5e76683af48e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:24.382374 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:24.383251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:24.383548 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:24.383829 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] PATH_INFO: `/v3/projects/fb17cb4253ac4cb68aa3357ca757c8ea` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:24.385042 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb17cb4253ac4cb68aa3357ca757c8ea)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:24.387001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:24.493094 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb17cb4253ac4cb68aa3357ca757c8ea', 'request_id': 'req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.498502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb17cb4253ac4cb68aa3357ca757c8ea', 'request_id': 'req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.504810 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb17cb4253ac4cb68aa3357ca757c8ea', 'request_id': 'req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.590222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] The token cache is being invalidate because project fb17cb4253ac4cb68aa3357ca757c8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:24.590515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.591141 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.592771 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1411/2840] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:24 2022] DELETE /identity/v3/projects/fb17cb4253ac4cb68aa3357ca757c8ea => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:24.596887 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352fca5-81c5-42f9-abd2-3145ec121e81 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:24.607643 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352fca5-81c5-42f9-abd2-3145ec121e81 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:24.611350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:24.664245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:24.666098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:24.666390 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:24.666692 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] PATH_INFO: `/v3/users/556171033520418a83b5f66a13f5f777` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:24.668132 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=556171033520418a83b5f66a13f5f777)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:24.670092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:24.786199 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556171033520418a83b5f66a13f5f777'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.787545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556171033520418a83b5f66a13f5f777'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.788152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556171033520418a83b5f66a13f5f777'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.804469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556171033520418a83b5f66a13f5f777'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.811353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-affc4658-e181-4800-93a3-628655165a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556171033520418a83b5f66a13f5f777'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:24.882389 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1430/2841] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:17:24 2022] DELETE /identity/v3/users/556171033520418a83b5f66a13f5f777 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:25.311969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:25.313449 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:25.327628 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f05f9-d84d-4f7c-a1ec-7c480a3e6bf7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:25.359633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:25.386483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:25.387378 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:25.387660 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:25.387961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] PATH_INFO: `/v3/projects/0c3bf8d94f05450f810b8e7e9d313834` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:25.389402 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c3bf8d94f05450f810b8e7e9d313834)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:25.391350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:25.489271 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c3bf8d94f05450f810b8e7e9d313834', 'request_id': 'req-abb06531-ab32-4768-baff-a1b0d57e7b25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:25.496806 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c3bf8d94f05450f810b8e7e9d313834', 'request_id': 'req-abb06531-ab32-4768-baff-a1b0d57e7b25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:25.502963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c3bf8d94f05450f810b8e7e9d313834', 'request_id': 'req-abb06531-ab32-4768-baff-a1b0d57e7b25'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:25.595619 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] The token cache is being invalidate because project 0c3bf8d94f05450f810b8e7e9d313834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:25.595845 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:25.596483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-abb06531-ab32-4768-baff-a1b0d57e7b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:25.598026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1412/2842] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:25 2022] DELETE /identity/v3/projects/0c3bf8d94f05450f810b8e7e9d313834 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:25.603382 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7188-4290-4786-bea3-0d1b9e6fb74e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:25.603867 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7188-4290-4786-bea3-0d1b9e6fb74e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:25.603976 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7188-4290-4786-bea3-0d1b9e6fb74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:25.623176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c23c7188-4290-4786-bea3-0d1b9e6fb74e None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:25.635482 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c23c7188-4290-4786-bea3-0d1b9e6fb74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:25.682895 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1431/2843] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:25.690515 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a18609-d076-4310-96ef-af73375348bd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:25.690948 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a18609-d076-4310-96ef-af73375348bd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:25.691267 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a18609-d076-4310-96ef-af73375348bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:25.710326 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-64a18609-d076-4310-96ef-af73375348bd None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:25.711601 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-64a18609-d076-4310-96ef-af73375348bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:25.719727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1413/2844] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:25.724635 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7188-4290-4786-bea3-0d1b9e6fb74e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:25.734342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7188-4290-4786-bea3-0d1b9e6fb74e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:25.738949 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:25.739654 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:25.740576 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:25.740849 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:25.741567 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:17:25.741871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:25.743503 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:25.757447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1432/2845] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:17:25.759612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a18609-d076-4310-96ef-af73375348bd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:25.769019 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a18609-d076-4310-96ef-af73375348bd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:25.773755 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe8f40-4866-491d-96e8-951c1d539a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:25.774397 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe8f40-4866-491d-96e8-951c1d539a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:25.774679 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe8f40-4866-491d-96e8-951c1d539a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:25.774955 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe8f40-4866-491d-96e8-951c1d539a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:25.775681 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe8f40-4866-491d-96e8-951c1d539a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:25.777549 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe8f40-4866-491d-96e8-951c1d539a45 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:25.877807 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1414/2846] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:25 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:25.878646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:25.889169 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5770a66-43d9-4135-aa5a-dc6632f75cd4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:25.898891 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:25.947965 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7eb4a-5dc7-4154-82ea-0701f711ebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:25.948627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7eb4a-5dc7-4154-82ea-0701f711ebe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:25.948904 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7eb4a-5dc7-4154-82ea-0701f711ebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:25.949175 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7eb4a-5dc7-4154-82ea-0701f711ebe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:25.949848 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7eb4a-5dc7-4154-82ea-0701f711ebe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:25.951769 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7eb4a-5dc7-4154-82ea-0701f711ebe9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:26.024357 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1433/2847] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:25 2022] POST /identity/v3/users => generated 282 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:26.028822 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe8f40-4866-491d-96e8-951c1d539a45 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:26.040027 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe8f40-4866-491d-96e8-951c1d539a45 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:26.044912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110ef7c-41a5-43de-853b-e1296139799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:26.045594 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110ef7c-41a5-43de-853b-e1296139799a admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:26.045908 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110ef7c-41a5-43de-853b-e1296139799a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:26.046185 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110ef7c-41a5-43de-853b-e1296139799a admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:26.046915 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110ef7c-41a5-43de-853b-e1296139799a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:26.048542 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110ef7c-41a5-43de-853b-e1296139799a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:26.059483 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1415/2848] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:26.063646 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7eb4a-5dc7-4154-82ea-0701f711ebe9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:26.074218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7eb4a-5dc7-4154-82ea-0701f711ebe9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:26.078926 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf9a27-958d-4832-9b16-d77aeecaf6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:26.079626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf9a27-958d-4832-9b16-d77aeecaf6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:26.082288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf9a27-958d-4832-9b16-d77aeecaf6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:26.082288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf9a27-958d-4832-9b16-d77aeecaf6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:26.082288 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf9a27-958d-4832-9b16-d77aeecaf6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:26.082579 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf9a27-958d-4832-9b16-d77aeecaf6f3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:26.093061 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1434/2849] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:26.097322 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110ef7c-41a5-43de-853b-e1296139799a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:26.108325 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110ef7c-41a5-43de-853b-e1296139799a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:26.112963 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:26.114222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:26.114222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:26.114440 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] PATH_INFO: `/v3/projects/b8a0410aa5f7419fa34aa233d432313e/users/7b0b2812db274beb87c7c5ff1c8fe5e9/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:26.129144 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a0410aa5f7419fa34aa233d432313e, user_id=7b0b2812db274beb87c7c5ff1c8fe5e9, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:26.131885 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:26.201745 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1416/2850] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:26 2022] PUT /identity/v3/projects/b8a0410aa5f7419fa34aa233d432313e/users/7b0b2812db274beb87c7c5ff1c8fe5e9/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:26.217088 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0fa79-443c-41a8-82c9-05e10156a89f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:26.217380 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0fa79-443c-41a8-82c9-05e10156a89f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:26.217655 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0fa79-443c-41a8-82c9-05e10156a89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:26.246176 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e7b0fa79-443c-41a8-82c9-05e10156a89f None None] MFA Rules not processed for user `7b0b2812db274beb87c7c5ff1c8fe5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:26.247429 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e7b0fa79-443c-41a8-82c9-05e10156a89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:26.313225 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1435/2851] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:26.325737 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:26.327574 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:26.384764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eeadd-7e3a-4040-8eeb-1168f4d511a8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:26.388387 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d86535-65bc-4127-ac55-a73475a7d492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:26.389034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d86535-65bc-4127-ac55-a73475a7d492 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:26.389321 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d86535-65bc-4127-ac55-a73475a7d492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:26.389602 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d86535-65bc-4127-ac55-a73475a7d492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:26.398563 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d86535-65bc-4127-ac55-a73475a7d492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:26.400013 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d86535-65bc-4127-ac55-a73475a7d492 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:26.408995 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1417/2852] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:31.143064 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0fa79-443c-41a8-82c9-05e10156a89f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:31.144844 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e7b0fa79-443c-41a8-82c9-05e10156a89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:31.208732 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0fa79-443c-41a8-82c9-05e10156a89f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:31.213818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:31.214680 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:31.214974 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:31.215269 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] PATH_INFO: `/v3/users/818dbd27888e486dae4114b182519932` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:31.216470 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=818dbd27888e486dae4114b182519932)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:31.218375 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:31.241423 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:31.397972 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818dbd27888e486dae4114b182519932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:31.398550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818dbd27888e486dae4114b182519932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:31.399164 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818dbd27888e486dae4114b182519932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:31.411583 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818dbd27888e486dae4114b182519932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:31.419107 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818dbd27888e486dae4114b182519932'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:31.487811 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1436/2853] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:17:31 2022] DELETE /identity/v3/users/818dbd27888e486dae4114b182519932 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:31.492274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2664f-8581-498e-9ccb-2f3ef6f04aa2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:31.492762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2664f-8581-498e-9ccb-2f3ef6f04aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:31.492872 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2664f-8581-498e-9ccb-2f3ef6f04aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:31.498761 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:31.513734 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-38f2664f-8581-498e-9ccb-2f3ef6f04aa2 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:31.515025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-38f2664f-8581-498e-9ccb-2f3ef6f04aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:31.577766 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1418/2854] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:31.588359 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:31.589798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:31.654927 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6524a-4c7b-44f5-8588-e8d32599ebd2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:31.658810 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5316fa7-6f6b-4c6b-84a8-5504080a1ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:31.659408 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5316fa7-6f6b-4c6b-84a8-5504080a1ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:31.659710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5316fa7-6f6b-4c6b-84a8-5504080a1ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:31.659954 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5316fa7-6f6b-4c6b-84a8-5504080a1ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:31.670319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5316fa7-6f6b-4c6b-84a8-5504080a1ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:31.671852 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5316fa7-6f6b-4c6b-84a8-5504080a1ca4 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:31.681516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1437/2855] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:32.079772 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2664f-8581-498e-9ccb-2f3ef6f04aa2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:32.079945 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-38f2664f-8581-498e-9ccb-2f3ef6f04aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:32.092323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2664f-8581-498e-9ccb-2f3ef6f04aa2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:32.097005 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:32.098087 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.098590 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.098786 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] PATH_INFO: `/v3/projects/0ee795307db94dc6b6e4b066ac5814a0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.100054 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee795307db94dc6b6e4b066ac5814a0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:32.101957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:32.213790 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee795307db94dc6b6e4b066ac5814a0', 'request_id': 'req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:32.219308 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee795307db94dc6b6e4b066ac5814a0', 'request_id': 'req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:32.225751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee795307db94dc6b6e4b066ac5814a0', 'request_id': 'req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:32.274743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] The token cache is being invalidate because project 0ee795307db94dc6b6e4b066ac5814a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:17:32.275034 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:32.275685 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-708b4e42-2bc8-4c05-9ef4-8097a0e4cd94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:17:32.277251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1419/2856] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:17:32 2022] DELETE /identity/v3/projects/0ee795307db94dc6b6e4b066ac5814a0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:32.282290 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eb6a8-c2ef-43b4-9d22-b1a92efe88f6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.282793 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eb6a8-c2ef-43b4-9d22-b1a92efe88f6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.282896 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eb6a8-c2ef-43b4-9d22-b1a92efe88f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.301307 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bd8eb6a8-c2ef-43b4-9d22-b1a92efe88f6 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:32.302516 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bd8eb6a8-c2ef-43b4-9d22-b1a92efe88f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:32.361544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1438/2857] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:32.370092 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b014-a618-431b-8ca0-59af8702fb76 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.370375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b014-a618-431b-8ca0-59af8702fb76 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.370651 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b014-a618-431b-8ca0-59af8702fb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.389314 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5b51b014-a618-431b-8ca0-59af8702fb76 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:32.390541 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5b51b014-a618-431b-8ca0-59af8702fb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:32.398674 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1420/2858] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:32.403090 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8eb6a8-c2ef-43b4-9d22-b1a92efe88f6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:32.412909 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8eb6a8-c2ef-43b4-9d22-b1a92efe88f6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:32.418143 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:32.418796 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.419077 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.419418 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.420138 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:17:32.420447 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:32.422028 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:32.434184 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1439/2859] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:17:32.439331 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51b014-a618-431b-8ca0-59af8702fb76 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:32.449243 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51b014-a618-431b-8ca0-59af8702fb76 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:32.454057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32004dd-c01d-451a-8060-51fb9d2955a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:32.454706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32004dd-c01d-451a-8060-51fb9d2955a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.454988 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32004dd-c01d-451a-8060-51fb9d2955a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.455298 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32004dd-c01d-451a-8060-51fb9d2955a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.456018 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32004dd-c01d-451a-8060-51fb9d2955a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:32.457941 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32004dd-c01d-451a-8060-51fb9d2955a7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:32.546488 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1421/2860] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:32 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:32.551102 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:32.562308 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94b33d-db72-4c6a-8571-3a95f5a954db admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:32.591110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:32.620081 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3e001-87b6-4864-ae24-925dc74bd6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:32.622596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e001-87b6-4864-ae24-925dc74bd6ed admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.622596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e001-87b6-4864-ae24-925dc74bd6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.622596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e001-87b6-4864-ae24-925dc74bd6ed admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.622596 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3e001-87b6-4864-ae24-925dc74bd6ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:32.624070 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3e001-87b6-4864-ae24-925dc74bd6ed admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:32.702448 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1440/2861] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:32 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:32.703216 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32004dd-c01d-451a-8060-51fb9d2955a7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:32.714642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32004dd-c01d-451a-8060-51fb9d2955a7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:32.719474 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e1ee-eab6-4221-931a-d25bd73313f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:32.721965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e1ee-eab6-4221-931a-d25bd73313f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.721965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e1ee-eab6-4221-931a-d25bd73313f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.721965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e1ee-eab6-4221-931a-d25bd73313f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.721965 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638e1ee-eab6-4221-931a-d25bd73313f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:32.722951 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638e1ee-eab6-4221-931a-d25bd73313f4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:32.734557 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1422/2862] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 04:17:32.738890 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3e001-87b6-4864-ae24-925dc74bd6ed admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:32.750445 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3e001-87b6-4864-ae24-925dc74bd6ed admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:32.756388 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f86c2-6376-4baa-b224-baa69c52abf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:32.757074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f86c2-6376-4baa-b224-baa69c52abf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.757352 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f86c2-6376-4baa-b224-baa69c52abf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.757627 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f86c2-6376-4baa-b224-baa69c52abf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.758353 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f86c2-6376-4baa-b224-baa69c52abf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:32.760004 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f86c2-6376-4baa-b224-baa69c52abf0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:32.770858 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1441/2863] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:32.775208 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e1ee-eab6-4221-931a-d25bd73313f4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:32.785693 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e1ee-eab6-4221-931a-d25bd73313f4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:32.791079 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:32.791994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.792284 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.792561 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] PATH_INFO: `/v3/projects/c3d3ef4d370f432bb537d528fdd0e718/users/7e3e60783dd542dfbc3672c15c2807e4/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.807283 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d3ef4d370f432bb537d528fdd0e718, user_id=7e3e60783dd542dfbc3672c15c2807e4, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:32.809866 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:32.917536 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1423/2864] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:32 2022] PUT /identity/v3/projects/c3d3ef4d370f432bb537d528fdd0e718/users/7e3e60783dd542dfbc3672c15c2807e4/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:17:32.924122 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59fcae-ffe9-428a-816d-2279b8124f10 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:32.924419 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59fcae-ffe9-428a-816d-2279b8124f10 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:32.924701 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59fcae-ffe9-428a-816d-2279b8124f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:32.942735 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-eb59fcae-ffe9-428a-816d-2279b8124f10 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:32.944231 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eb59fcae-ffe9-428a-816d-2279b8124f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:33.007797 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1442/2865] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:33.018480 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:33.023301 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:33.046962 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:33.082878 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc70f9-51cb-406f-9844-b8a6d2492267 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:33.087610 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaba591-0198-4067-909b-5963ce2c24b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:33.088258 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaba591-0198-4067-909b-5963ce2c24b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:33.088535 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaba591-0198-4067-909b-5963ce2c24b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:33.088912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaba591-0198-4067-909b-5963ce2c24b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:33.098725 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaba591-0198-4067-909b-5963ce2c24b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:33.100375 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaba591-0198-4067-909b-5963ce2c24b1 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:33.110803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1424/2866] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:38.564530 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697081-09b1-4fd8-b663-5a17570cb651 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:38.565074 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697081-09b1-4fd8-b663-5a17570cb651 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:38.565177 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697081-09b1-4fd8-b663-5a17570cb651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:38.597422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-32697081-09b1-4fd8-b663-5a17570cb651 None None] MFA Rules not processed for user `7e3e60783dd542dfbc3672c15c2807e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:38.598586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-32697081-09b1-4fd8-b663-5a17570cb651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:38.807546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1443/2867] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:38.812170 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaba591-0198-4067-909b-5963ce2c24b1 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:38.824808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaba591-0198-4067-909b-5963ce2c24b1 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:38.829631 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672a3c7-f42e-4f72-bbc2-90fb17cfc9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:38.830287 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a3c7-f42e-4f72-bbc2-90fb17cfc9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:38.830577 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a3c7-f42e-4f72-bbc2-90fb17cfc9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:38.830849 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a3c7-f42e-4f72-bbc2-90fb17cfc9a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:38.831584 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672a3c7-f42e-4f72-bbc2-90fb17cfc9a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:38.833465 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672a3c7-f42e-4f72-bbc2-90fb17cfc9a2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:38.946053 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1425/2868] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:17:38 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 04:17:38.950158 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32697081-09b1-4fd8-b663-5a17570cb651 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:38.962161 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32697081-09b1-4fd8-b663-5a17570cb651 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:39.001818 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:39.020526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1fc61-0cf8-4276-8943-bd842f15ed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:39.021195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1fc61-0cf8-4276-8943-bd842f15ed10 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:39.021480 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1fc61-0cf8-4276-8943-bd842f15ed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:39.021759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1fc61-0cf8-4276-8943-bd842f15ed10 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:39.023133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1fc61-0cf8-4276-8943-bd842f15ed10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:39.025092 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1fc61-0cf8-4276-8943-bd842f15ed10 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:39.104645 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1444/2869] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:17:38 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:17:39.108882 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672a3c7-f42e-4f72-bbc2-90fb17cfc9a2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:39.119286 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672a3c7-f42e-4f72-bbc2-90fb17cfc9a2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:39.124008 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87241fca-15e0-4e5a-aecd-5fdb6e409cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:39.124667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241fca-15e0-4e5a-aecd-5fdb6e409cff admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:39.124957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241fca-15e0-4e5a-aecd-5fdb6e409cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:39.125232 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241fca-15e0-4e5a-aecd-5fdb6e409cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:39.125971 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87241fca-15e0-4e5a-aecd-5fdb6e409cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:39.127581 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87241fca-15e0-4e5a-aecd-5fdb6e409cff admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:39.139637 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1426/2870] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:39.143809 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1fc61-0cf8-4276-8943-bd842f15ed10 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:39.155293 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1fc61-0cf8-4276-8943-bd842f15ed10 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:39.161822 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c041b34-55c1-4095-a591-e89b5b3b2990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:39.163033 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c041b34-55c1-4095-a591-e89b5b3b2990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:39.163350 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c041b34-55c1-4095-a591-e89b5b3b2990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:39.163629 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c041b34-55c1-4095-a591-e89b5b3b2990 admin admin] PATH_INFO: `/v3/projects/76accb2b71014a11a7193647c7ec7e02/users/39aa5cf2ebc34a829301281990fa9b8a/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:39.180162 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c041b34-55c1-4095-a591-e89b5b3b2990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76accb2b71014a11a7193647c7ec7e02, user_id=39aa5cf2ebc34a829301281990fa9b8a, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:39.182871 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c041b34-55c1-4095-a591-e89b5b3b2990 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:39.260319 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1445/2871] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:39 2022] PUT /identity/v3/projects/76accb2b71014a11a7193647c7ec7e02/users/39aa5cf2ebc34a829301281990fa9b8a/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:39.264052 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87241fca-15e0-4e5a-aecd-5fdb6e409cff admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:39.274936 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87241fca-15e0-4e5a-aecd-5fdb6e409cff admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:39.351858 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e11e72-fd3d-4d5a-9843-ed1ba9a3b64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:39.352538 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e11e72-fd3d-4d5a-9843-ed1ba9a3b64e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:39.352947 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e11e72-fd3d-4d5a-9843-ed1ba9a3b64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:39.353222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e11e72-fd3d-4d5a-9843-ed1ba9a3b64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:39.354028 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e11e72-fd3d-4d5a-9843-ed1ba9a3b64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:39.355661 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e11e72-fd3d-4d5a-9843-ed1ba9a3b64e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:39.370044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1427/2872] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:17:39.371287 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c041b34-55c1-4095-a591-e89b5b3b2990 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:39.385066 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c041b34-55c1-4095-a591-e89b5b3b2990 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:39.398152 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:39.399300 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:39.399587 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:39.399868 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] PATH_INFO: `/v3/projects/76accb2b71014a11a7193647c7ec7e02/users/39aa5cf2ebc34a829301281990fa9b8a/roles/2e3fcdfa6cc7468b840e70684e783ca1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:39.401935 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=76accb2b71014a11a7193647c7ec7e02, user_id=39aa5cf2ebc34a829301281990fa9b8a, role_id=2e3fcdfa6cc7468b840e70684e783ca1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:39.404610 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:39.415765 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1446/2873] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:17:39 2022] PUT /identity/v3/projects/76accb2b71014a11a7193647c7ec7e02/users/39aa5cf2ebc34a829301281990fa9b8a/roles/2e3fcdfa6cc7468b840e70684e783ca1 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:17:45.374808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1caf28-d60b-47c9-98fc-b27643fad8b9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:45.375392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1caf28-d60b-47c9-98fc-b27643fad8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:45.375392 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1caf28-d60b-47c9-98fc-b27643fad8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:45.405025 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1d1caf28-d60b-47c9-98fc-b27643fad8b9 None None] MFA Rules not processed for user `39aa5cf2ebc34a829301281990fa9b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:45.409348 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1d1caf28-d60b-47c9-98fc-b27643fad8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:45.476586 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1428/2874] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:17:45.493085 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:17:45.494864 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:45.608302 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c396f-d35e-4dd3-b3e9-4977fed45afc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:17:45.615142 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:17:45.616362 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:45.616625 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:45.616952 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:45.678923 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:17:45.680581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:17:45.696335 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1447/2875] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:17:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:17:48.542909 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a1fc-02eb-4c02-ae9f-40ab2211c2c7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:17:48.543659 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a1fc-02eb-4c02-ae9f-40ab2211c2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:17:48.543659 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a1fc-02eb-4c02-ae9f-40ab2211c2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:17:48.562949 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6784a1fc-02eb-4c02-ae9f-40ab2211c2c7 None None] MFA Rules not processed for user `7e3e60783dd542dfbc3672c15c2807e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:17:48.564251 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6784a1fc-02eb-4c02-ae9f-40ab2211c2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:17:48.572368 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1429/2876] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:17:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 04:17:55.686547 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:17:58.557353 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:18:09.904666 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:09.906120 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:09.970717 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f319611-4e48-45fa-8a27-cde8b058a663 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:09.981723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:09.981723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:09.981723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:09.981723 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] PATH_INFO: `/v3/users/7b0b2812db274beb87c7c5ff1c8fe5e9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:09.982133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0b2812db274beb87c7c5ff1c8fe5e9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:09.982133 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:10.163187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0b2812db274beb87c7c5ff1c8fe5e9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:10.163804 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0b2812db274beb87c7c5ff1c8fe5e9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:10.164415 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0b2812db274beb87c7c5ff1c8fe5e9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:10.177321 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0b2812db274beb87c7c5ff1c8fe5e9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:10.184985 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0b2812db274beb87c7c5ff1c8fe5e9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:10.259292 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1448/2877] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:18:09 2022] DELETE /identity/v3/users/7b0b2812db274beb87c7c5ff1c8fe5e9 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:18:10.263848 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e2609-de9c-4251-9d6d-1095d532029a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:10.264414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e2609-de9c-4251-9d6d-1095d532029a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:10.264414 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e2609-de9c-4251-9d6d-1095d532029a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:10.283078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-614e2609-de9c-4251-9d6d-1095d532029a None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:18:10.284359 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-614e2609-de9c-4251-9d6d-1095d532029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:10.344830 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1430/2878] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:18:10.354570 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:10.356089 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:10.419562 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3613f0c-cec7-41d1-87ab-804685c4cb1b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:10.423929 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27106-0f8e-4c66-b188-8a7a4a9641b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:10.432125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27106-0f8e-4c66-b188-8a7a4a9641b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:10.432125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27106-0f8e-4c66-b188-8a7a4a9641b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:10.432125 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27106-0f8e-4c66-b188-8a7a4a9641b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:10.434916 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe27106-0f8e-4c66-b188-8a7a4a9641b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:10.436421 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe27106-0f8e-4c66-b188-8a7a4a9641b5 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:10.446925 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1449/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:18:10.871421 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e2609-de9c-4251-9d6d-1095d532029a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:10.872851 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-614e2609-de9c-4251-9d6d-1095d532029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:10.884861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e2609-de9c-4251-9d6d-1095d532029a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:10.891643 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:10.892706 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:10.892994 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:10.893272 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] PATH_INFO: `/v3/projects/b8a0410aa5f7419fa34aa233d432313e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:10.894636 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a0410aa5f7419fa34aa233d432313e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:10.896886 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:11.109943 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a0410aa5f7419fa34aa233d432313e', 'request_id': 'req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:11.115656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a0410aa5f7419fa34aa233d432313e', 'request_id': 'req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:11.121912 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a0410aa5f7419fa34aa233d432313e', 'request_id': 'req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:11.223118 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] The token cache is being invalidate because project b8a0410aa5f7419fa34aa233d432313e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:18:11.223435 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:11.224090 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cdcdd0d1-0c2d-483d-9f4b-73e1109060ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:11.225633 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1431/2880] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:18:10 2022] DELETE /identity/v3/projects/b8a0410aa5f7419fa34aa233d432313e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:18:11.230604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c50d-e959-47b1-823c-fc432ad65f23 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.231278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c50d-e959-47b1-823c-fc432ad65f23 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.231278 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c50d-e959-47b1-823c-fc432ad65f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.249768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-baa8c50d-e959-47b1-823c-fc432ad65f23 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:18:11.253063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-baa8c50d-e959-47b1-823c-fc432ad65f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:11.269963 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:18:11.312711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1450/2881] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:18:11.320803 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474005b4-c979-4957-b809-a5992b7ddab1 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.321323 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474005b4-c979-4957-b809-a5992b7ddab1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.356333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474005b4-c979-4957-b809-a5992b7ddab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.356333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-474005b4-c979-4957-b809-a5992b7ddab1 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:18:11.356333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-474005b4-c979-4957-b809-a5992b7ddab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:11.356333 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1432/2882] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:18:11.359550 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8c50d-e959-47b1-823c-fc432ad65f23 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:11.369349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8c50d-e959-47b1-823c-fc432ad65f23 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:11.374543 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:11.375213 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.375677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.375977 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.376700 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 04:18:11.377005 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:11.378594 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:11.395467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1451/2883] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 1 04:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 04:18:11.396929 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474005b4-c979-4957-b809-a5992b7ddab1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:11.416743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474005b4-c979-4957-b809-a5992b7ddab1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:11.421824 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235b8e0-200d-459d-8ea0-cdc177a03ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:11.425195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b8e0-200d-459d-8ea0-cdc177a03ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.425195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b8e0-200d-459d-8ea0-cdc177a03ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.425195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b8e0-200d-459d-8ea0-cdc177a03ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.425195 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235b8e0-200d-459d-8ea0-cdc177a03ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:11.427180 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235b8e0-200d-459d-8ea0-cdc177a03ee7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:11.524570 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1433/2884] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 1 04:18:11 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:18:11.528688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:11.548264 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1d88e-513c-4ac5-938d-ad80e815bd44 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:11.620131 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080d48f-7dc0-4ea9-8d0b-0c50da33cc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:11.620677 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080d48f-7dc0-4ea9-8d0b-0c50da33cc3f admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.620959 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080d48f-7dc0-4ea9-8d0b-0c50da33cc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.621234 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080d48f-7dc0-4ea9-8d0b-0c50da33cc3f admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.621918 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080d48f-7dc0-4ea9-8d0b-0c50da33cc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:11.624422 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080d48f-7dc0-4ea9-8d0b-0c50da33cc3f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:11.731537 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1452/2885] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 1 04:18:11 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 04:18:11.736379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235b8e0-200d-459d-8ea0-cdc177a03ee7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:11.746642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235b8e0-200d-459d-8ea0-cdc177a03ee7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:11.751779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708424-a62f-4307-b686-145f4b4ee76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:11.752446 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708424-a62f-4307-b686-145f4b4ee76e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.752731 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708424-a62f-4307-b686-145f4b4ee76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.753009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708424-a62f-4307-b686-145f4b4ee76e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.753743 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3708424-a62f-4307-b686-145f4b4ee76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:11.755350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3708424-a62f-4307-b686-145f4b4ee76e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:11.767369 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1434/2886] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:18:11.772129 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080d48f-7dc0-4ea9-8d0b-0c50da33cc3f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:11.782919 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080d48f-7dc0-4ea9-8d0b-0c50da33cc3f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:11.787713 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f0b18-6086-4411-b256-07474a6096a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:11.788387 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f0b18-6086-4411-b256-07474a6096a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.788670 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f0b18-6086-4411-b256-07474a6096a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.788948 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f0b18-6086-4411-b256-07474a6096a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.789685 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f0b18-6086-4411-b256-07474a6096a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:11.791342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f0b18-6086-4411-b256-07474a6096a2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:11.802544 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1453/2887] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 1 04:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 04:18:11.806841 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708424-a62f-4307-b686-145f4b4ee76e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:11.817904 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708424-a62f-4307-b686-145f4b4ee76e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:11.822871 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:11.823934 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.824224 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.824503 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] PATH_INFO: `/v3/projects/b6b1ede4ce9246b682faa96a24013f1b/users/2d6000f196d84898bf22fe2dc7a8fd13/roles/2e43acd667e44e0bbba45926e652a64f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.840694 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b1ede4ce9246b682faa96a24013f1b, user_id=2d6000f196d84898bf22fe2dc7a8fd13, role_id=2e43acd667e44e0bbba45926e652a64f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:11.844026 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:11.939310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1435/2888] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 1 04:18:11 2022] PUT /identity/v3/projects/b6b1ede4ce9246b682faa96a24013f1b/users/2d6000f196d84898bf22fe2dc7a8fd13/roles/2e43acd667e44e0bbba45926e652a64f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:18:11.950128 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5b58-9d12-4628-91bb-387b2c12dd37 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:11.950413 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5b58-9d12-4628-91bb-387b2c12dd37 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:11.950688 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5b58-9d12-4628-91bb-387b2c12dd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:11.977508 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9e5e5b58-9d12-4628-91bb-387b2c12dd37 None None] MFA Rules not processed for user `2d6000f196d84898bf22fe2dc7a8fd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:18:11.978716 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9e5e5b58-9d12-4628-91bb-387b2c12dd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:12.045619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1454/2889] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:18:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:18:12.056499 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:12.058182 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:12.080335 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:18:12.116274 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094c4b1-ad01-4cd1-88ec-ea373e0c0e4b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:12.120010 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fee85d-c0ea-4c1d-b7d8-f70a82f03e97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8a755e84f04c9590c9c8d57781d2a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:12.120658 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee85d-c0ea-4c1d-b7d8-f70a82f03e97 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:12.120939 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee85d-c0ea-4c1d-b7d8-f70a82f03e97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:12.121215 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee85d-c0ea-4c1d-b7d8-f70a82f03e97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:12.129823 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fee85d-c0ea-4c1d-b7d8-f70a82f03e97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:12.131317 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fee85d-c0ea-4c1d-b7d8-f70a82f03e97 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:12.140247 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1436/2890] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:18:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:18:22.042110 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:18:22.703721 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=17 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:18:23.457696 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e5b58-9d12-4628-91bb-387b2c12dd37 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:23.523474 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e5b58-9d12-4628-91bb-387b2c12dd37 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:23.528211 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:23.529073 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:23.529349 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:23.529616 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] PATH_INFO: `/v3/users/2d6000f196d84898bf22fe2dc7a8fd13` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:23.530955 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d6000f196d84898bf22fe2dc7a8fd13)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:23.532888 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:23.698720 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d6000f196d84898bf22fe2dc7a8fd13'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:23.699385 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d6000f196d84898bf22fe2dc7a8fd13'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:23.700003 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d6000f196d84898bf22fe2dc7a8fd13'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:23.718623 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d6000f196d84898bf22fe2dc7a8fd13'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:23.725971 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d6000f196d84898bf22fe2dc7a8fd13'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:23.840578 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1455/2891] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:18:23 2022] DELETE /identity/v3/users/2d6000f196d84898bf22fe2dc7a8fd13 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:18:23.845604 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cbee4-cfd5-49f0-a0a3-84f5319d3567 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:23.846083 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cbee4-cfd5-49f0-a0a3-84f5319d3567 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:23.846191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cbee4-cfd5-49f0-a0a3-84f5319d3567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:23.864564 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-245cbee4-cfd5-49f0-a0a3-84f5319d3567 None None] MFA Rules not processed for user `d62406877bb84126838f185cae41e3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 04:18:23.865751 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-245cbee4-cfd5-49f0-a0a3-84f5319d3567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:23.925896 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1437/2892] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 1 04:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 04:18:23.935691 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:23.937243 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:23.996218 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f6959-abaa-4686-b0e0-02708a545b29 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:23.999839 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:24.000484 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:24.000763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:24.001036 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:24.010711 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:24.012314 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:24.024124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1456/2893] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:18:24.425310 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cbee4-cfd5-49f0-a0a3-84f5319d3567 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:24.426902 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-245cbee4-cfd5-49f0-a0a3-84f5319d3567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:24.439379 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cbee4-cfd5-49f0-a0a3-84f5319d3567 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:24.445416 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:24.446508 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:24.447762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:24.447762 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] PATH_INFO: `/v3/projects/b6b1ede4ce9246b682faa96a24013f1b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:24.452111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b1ede4ce9246b682faa96a24013f1b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:24.452111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:24.551120 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b1ede4ce9246b682faa96a24013f1b', 'request_id': 'req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:24.556656 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b1ede4ce9246b682faa96a24013f1b', 'request_id': 'req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:24.563481 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b1ede4ce9246b682faa96a24013f1b', 'request_id': 'req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:24.645927 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] The token cache is being invalidate because project b6b1ede4ce9246b682faa96a24013f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:18:24.646222 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:24.646869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:18:24.648443 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1438/2894] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:18:24 2022] DELETE /identity/v3/projects/b6b1ede4ce9246b682faa96a24013f1b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:18:25.849459 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:25.851057 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:25.863063 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204254e0-91cd-4db6-bfd9-ceccbc4394fd service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:25.881882 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:18:25.912139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:25.912788 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:25.913071 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:25.913407 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:25.915245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:25.979245 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:25.980693 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:25.990744 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1457/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:18:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:18:34.642741 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:18:35.986588 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=40 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:18:45.307854 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:18:45.309313 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:45.414657 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcfbd3-fd86-450a-91a3-71f5fcaf0d50 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:18:45.420056 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:18:45.420714 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:18:45.421001 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:18:45.421282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:18:45.423131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:18:45.435764 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:18:45.437667 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:18:45.448145 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1439/2896] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:18:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:18:55.443502 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=41 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:20:23.515853 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:20:23.517370 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:20:23.528914 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bdb53-2710-44bd-8015-461fe1389105 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:20:23.532581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:20:23.533223 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:20:23.533527 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:20:23.533798 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:20:23.536331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:20:23.615181 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:20:23.616710 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:20:23.628526 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1458/2897] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:20:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:20:24.803057 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:20:24.804582 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:20:24.816078 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea54cb-00a3-4c11-b120-76a9af54a8a4 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:20:24.820273 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:20:24.820924 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:20:24.821211 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:20:24.821492 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:20:24.823327 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:20:24.837805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:20:24.841147 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:20:24.851033 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1440/2898] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:20:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:20:33.622892 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=34 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:20:33.747342 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:20:33.757725 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb340b9a-f066-401c-9dc7-962b54d8d704 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:20:33.761495 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:20:33.762156 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:20:33.762436 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:20:33.762712 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:20:33.764582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:20:33.775759 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:20:33.777195 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:20:33.786276 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1459/2899] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:20:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:20:34.845458 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:20:38.181580 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:20:38.190505 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6468d5-4a28-4097-9e99-46b6c6708389 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:20:38.194302 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:20:38.194957 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:20:38.195521 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:20:38.195797 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:20:38.197780 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:20:38.210134 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:20:38.211278 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:20:38.221407 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1441/2900] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:20:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:20:38.313490 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:20:38.323545 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623ba68-455c-428e-9344-9f1798cb23c7 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:20:38.327467 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0beff646206e446386d91537df9eaa67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:20:38.328124 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:20:38.328403 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:20:38.328678 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:20:38.330518 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:20:38.345369 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:20:38.346532 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:20:38.358590 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1460/2901] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:20:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:20:48.217085 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:20:48.353454 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=12 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:22:03.432237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:22:03.432237 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:22:03.445740 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792a20c-6daa-4927-872e-48b8950a5f86 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:22:03.452044 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbd31b58571b476bad12735ff7445d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:22:03.452779 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:22:03.453161 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:22:03.453553 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:22:03.455808 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:22:03.469418 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:22:03.471136 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:22:03.484571 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1442/2902] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:22:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:22:12.124546 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:22:12.137575 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf68fa-5b01-470d-b682-4dd82c4cd12d service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:22:12.143130 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:22:12.144080 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:22:12.144559 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:22:12.145019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:22:12.147111 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:22:12.160582 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:22:12.162173 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:22:12.173581 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1461/2903] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:22:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:22:13.479562 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:22:15.376235 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:22:15.387861 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c1fa4-e08e-4cd6-b3a1-5a405486ebbe service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:22:15.393154 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:22:15.393947 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:22:15.394334 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:22:15.394715 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:22:15.396804 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:22:15.409206 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:22:15.410869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:22:15.421805 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1443/2904] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:22:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:22:15.512099 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:22:15.513602 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:22:15.579476 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea14e5-e4eb-47be-b165-2e119cd83757 service placement] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:22:15.584170 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:22:15.585038 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:22:15.585320 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:22:15.585595 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] PATH_INFO: `/v3/users/7e3e60783dd542dfbc3672c15c2807e4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:22:15.586768 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3e60783dd542dfbc3672c15c2807e4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:22:15.588703 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:22:15.876563 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3e60783dd542dfbc3672c15c2807e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:15.877134 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3e60783dd542dfbc3672c15c2807e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:15.877763 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3e60783dd542dfbc3672c15c2807e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:15.889400 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3e60783dd542dfbc3672c15c2807e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:15.896604 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3e60783dd542dfbc3672c15c2807e4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:15.988331 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1462/2905] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:22:15 2022] DELETE /identity/v3/users/7e3e60783dd542dfbc3672c15c2807e4 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:22:16.232989 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:22:16.244111 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287615-1efd-4aa9-836d-8a06854e45cc service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:22:16.304846 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:22:16.305682 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:22:16.305969 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:22:16.306249 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] PATH_INFO: `/v3/projects/c3d3ef4d370f432bb537d528fdd0e718` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:22:16.307466 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d3ef4d370f432bb537d528fdd0e718)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:22:16.309350 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:22:16.440024 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d3ef4d370f432bb537d528fdd0e718', 'request_id': 'req-a6ec2120-c044-4271-a7f5-220612dc4cde'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.446470 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d3ef4d370f432bb537d528fdd0e718', 'request_id': 'req-a6ec2120-c044-4271-a7f5-220612dc4cde'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.452627 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d3ef4d370f432bb537d528fdd0e718', 'request_id': 'req-a6ec2120-c044-4271-a7f5-220612dc4cde'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.514016 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] The token cache is being invalidate because project c3d3ef4d370f432bb537d528fdd0e718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:22:16.514302 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.514961 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.516663 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1444/2906] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:22:16 2022] DELETE /identity/v3/projects/c3d3ef4d370f432bb537d528fdd0e718 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:22:16.520517 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:22:16.522155 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:22:16.535043 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67da862-ffdb-4533-9ee9-34ad233a12fb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:22:16.761511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:22:16.761511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:22:16.761511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:22:16.761511 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] PATH_INFO: `/v3/users/39aa5cf2ebc34a829301281990fa9b8a` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:22:16.762820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39aa5cf2ebc34a829301281990fa9b8a)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:22:16.762820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:22:16.762820 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:22:16.895821 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39aa5cf2ebc34a829301281990fa9b8a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.896502 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39aa5cf2ebc34a829301281990fa9b8a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.897098 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39aa5cf2ebc34a829301281990fa9b8a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.909317 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39aa5cf2ebc34a829301281990fa9b8a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.916619 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39aa5cf2ebc34a829301281990fa9b8a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:16.984469 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1463/2907] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 1 04:22:16 2022] DELETE /identity/v3/users/39aa5cf2ebc34a829301281990fa9b8a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 04:22:17.209894 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:22:17.211363 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:22:17.225193 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec2120-c044-4271-a7f5-220612dc4cde admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:22:17.280869 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd62406877bb84126838f185cae41e3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e12b5dbdf665489f80381a9cc1ab6bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:22:17.281727 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:22:17.282009 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:22:17.282282 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] PATH_INFO: `/v3/projects/76accb2b71014a11a7193647c7ec7e02` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:22:17.287906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76accb2b71014a11a7193647c7ec7e02)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:22:17.287906 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:22:17.306573 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:22:17.397434 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76accb2b71014a11a7193647c7ec7e02', 'request_id': 'req-bb5935ed-d409-4197-b973-30af0345142b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:17.402982 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76accb2b71014a11a7193647c7ec7e02', 'request_id': 'req-bb5935ed-d409-4197-b973-30af0345142b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:17.409131 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76accb2b71014a11a7193647c7ec7e02', 'request_id': 'req-bb5935ed-d409-4197-b973-30af0345142b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:17.485642 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] The token cache is being invalidate because project 76accb2b71014a11a7193647c7ec7e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 04:22:17.486347 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:17.486612 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb5935ed-d409-4197-b973-30af0345142b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 04:22:17.488191 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1445/2908] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 1 04:22:17 2022] DELETE /identity/v3/projects/76accb2b71014a11a7193647c7ec7e02 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 04:22:27.094167 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:22:27.482596 dv-cinder-846842-8-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 04:23:50.257626 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 04:23:50.259021 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:23:50.327187 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef212b-e365-4b16-888c-abb5801e23a9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 04:23:50.389295 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91f5c6-e5e8-4132-ba45-cf213a9f61a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f408d50bfc3a4366aabb1cdf86da7935', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bcddee341bf4069b0154f2eceb30cf3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 04:23:50.389930 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f5c6-e5e8-4132-ba45-cf213a9f61a1 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 04:23:50.390215 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f5c6-e5e8-4132-ba45-cf213a9f61a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 04:23:50.390491 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f5c6-e5e8-4132-ba45-cf213a9f61a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 04:23:50.392325 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6d91f5c6-e5e8-4132-ba45-cf213a9f61a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 04:23:50.404019 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91f5c6-e5e8-4132-ba45-cf213a9f61a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 04:23:50.405586 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91f5c6-e5e8-4132-ba45-cf213a9f61a1 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 04:23:50.416139 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1464/2909] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 1 04:23:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 04:24:00.411862 dv-cinder-846842-8-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=58 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}